A model is still running

A model is still running or downloading. Leaving will stop the current operation. Stay on this page to let it finish.

Skip to content
LocalAI.toolsBeta

Join the waitlist

Get an email when the new apps and model runs launch.

No account needed.

Granite 4.0

Local chat, drafting and text extraction using IBM Granite 4.0 Micro.

Desktop inferencePrivate · Runs on your computer
About this model

Local text generation with IBM Granite 4.0 Dense 1B Base. Verified for short English prose and Python continuation with a checked next word or code token.

Created by
IBM Granite
License
Apache-2.0
Model released
Oct 28, 2025
Source repository created
Oct 7, 2025
Model weights · BF16
3.3 GB

Best for

  • Continue short prose prefixes
  • Continue short code prefixes

Keep in mind

  • Raw pretrained text continuation. This adapter accepts a literal text prefix.
  • Ambiguous prefixes can yield unrelated, inaccurate or unsafe text. Verify claims and test generated code.
  • Quality checks cover only a few English prose and Python prefixes. Instructions, tool use, other modalities and long contexts remain unverified.
  • The combined prefix and generation budget is limited to 8192 tokens.

Processed on your deviceYour input is processed locally in the desktop app. Model and runtime files download from external hosts.

Target platforms · verification required

  • macos · aarch64, x86_64
  • windows · x86_64

Recommended configuration

BF16 · mps (verified platform only)

Sources & technical details
Original model
https://huggingface.co/ibm-granite/granite-4.0-1b-base
Release verification
Verified
Release evidence
The original checkpoint's model card names its release date.
Source updated
Jun 12, 2026
Native model files
ibm-granite/granite-4.0-1b-base
Files updated
Jun 12, 2026
Artifact revision
15148e99f8ca689325e2067ba282b23dec6670c7

Desktop app

Sample result

Granite 4.0 Dense 1B Base

Run Granite 4.0 on your computer

Download the desktop app to load this model, provide input and view results locally.

Public desktop downloads are coming soon.

Device requirements

macOS Apple SiliconCPU / FP32Tested16.0 GiBRAM estimate
macOS Apple SiliconMPS / BF16Tested16.0 GiBRAM estimate
Windows Intel / AMDCPU / FP32Unverified16.0 GiBRAM estimate
macOS Intel / AMDCPU / FP32Unverified16.0 GiBRAM estimate
If your device falls short

MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.

GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.

PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.

Benchmark evidence

Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.

Granite 4.0 Dense 1B Base

24 configurations tested
Passed21
Failed3
TestedNov 7, 2026
Runtimetransformers 5.22.1

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
BF16mpsAcceleratedPassedFastmacos · aarch64191 ms / 1453 ms
BF16mpsAcceleratedPassedFastmacos · aarch64191 ms / 1453 ms
BF16mpsAcceleratedPassedFastmacos · aarch64194 ms / 1443 ms
BF16mpsAcceleratedPassedFastmacos · aarch64195 ms / 1443 ms
BF16mpsAcceleratedPassedFastmacos · aarch64200 ms / 1453 ms
BF16mpsAcceleratedPassedFastmacos · aarch64207 ms / 1443 ms
BF16mpsAcceleratedPassedFastmacos · aarch64329 ms / 1652 ms
BF16mpsAcceleratedPassedModeratemacos · aarch64580 ms / 1652 ms
BF16mpsAcceleratedPassedModeratemacos · aarch64797 ms / 1652 ms
FP32cpuPassedModeratemacos · aarch64691 ms / 1263 ms
FP32cpuPassedModeratemacos · aarch64705 ms / 1263 ms
FP32cpuPassedModeratemacos · aarch64787 ms / 1263 ms
FP32cpuPassedModeratemacos · aarch64837 ms / 1347 ms
FP32cpuPassedModeratemacos · aarch64840 ms / 1216 ms
FP32cpuPassedModeratemacos · aarch64841 ms / 1216 ms
FP32cpuPassedModeratemacos · aarch64870 ms / 1216 ms
FP32cpuPassedModeratemacos · aarch64914 ms / 1347 ms
FP32cpuPassedModeratemacos · aarch64916 ms / 1462 ms
FP32cpuPassedModeratemacos · aarch64945 ms / 1462 ms
FP32cpuPassedModeratemacos · aarch641054 ms / 1347 ms
FP32cpuPassedModeratemacos · aarch641155 ms / 1462 ms
FP16mpsFailed—macos · aarch64575 ms / 2381 ms
FP16mpsFailed—macos · aarch64376 ms / 2381 ms
FP16mpsFailed—macos · aarch64397 ms / 2381 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-07T12:43:40.917651+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 5.22.1
Artifact revision
15148e99f8ca689325e2067ba282b23dec6670c7
Fixture
{
  "file": "model-samples/desktop/granite4/granite-4.0-1b-base-native-geography.json",
  "size": 152,
  "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef",
  "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21",
  "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-1b-base-native-geography.json"
}
Rerun when
Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
  {
    "precision": "fp16",
    "device": "mps",
    "status": "fail",
    "quality": {
      "expectedText": false,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": false,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 2380.7257910011685,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 322
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 576.3418330025161,
      "inferenceMs": 575.4771669962793,
      "tokensPerSecond": 13.901507233998954,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 4323147776,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 1692270592,
      "cpuThreads": 4,
      "totalMs": 2957.0676240036846,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "fail",
    "quality": {
      "expectedText": false,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 2380.7257910011685,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 322
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 376.92220799362985,
      "inferenceMs": 375.76629200339084,
      "tokensPerSecond": 21.289828731971014,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 4323229696,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 1702805504,
      "cpuThreads": 4,
      "totalMs": 2757.6479989947984,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "fail",
    "quality": {
      "expectedText": false,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 2380.7257910011685,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 322
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 398.6867089988664,
      "inferenceMs": 397.0865839946782,
      "tokensPerSecond": 20.146739583896938,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 4323328000,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 1713799168,
      "cpuThreads": 4,
      "totalMs": 2779.412500000035,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1462.1262500004377,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 916.7027500006952,
      "inferenceMs": 916.3787919969764,
      "tokensPerSecond": 8.730014345450279,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6965641216,
      "cpuThreads": 4,
      "totalMs": 2378.829000001133,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1462.1262500004377,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1155.3825420050998,
      "inferenceMs": 1155.0865829995018,
      "tokensPerSecond": 6.925887736680126,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6968016896,
      "cpuThreads": 4,
      "totalMs": 2617.5087920055375,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1462.1262500004377,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 945.3702499959036,
      "inferenceMs": 945.0021250013378,
      "tokensPerSecond": 8.465589429218134,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6968918016,
      "cpuThreads": 4,
      "totalMs": 2407.4964999963413,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1216.2165830013691,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 869.9214169973857,
      "inferenceMs": 869.663124998624,
      "tokensPerSecond": 9.198964254133069,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6961905664,
      "cpuThreads": 4,
      "totalMs": 2086.137999998755,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1216.2165830013691,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 840.6399170053191,
      "inferenceMs": 840.3959159986698,
      "tokensPerSecond": 9.519322795010659,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6964297728,
      "cpuThreads": 4,
      "totalMs": 2056.8565000066883,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1216.2165830013691,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 841.5045000001555,
      "inferenceMs": 841.2319589988329,
      "tokensPerSecond": 9.509862190115745,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6965542912,
      "cpuThreads": 4,
      "totalMs": 2057.7210830015247,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1346.6786249991856,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1053.899457998341,
      "inferenceMs": 1053.6382920035976,
      "tokensPerSecond": 7.592738476491023,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6960054272,
      "cpuThreads": 4,
      "totalMs": 2400.5780829975265,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1346.6786249991856,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 913.8644169943291,
      "inferenceMs": 913.5276250017341,
      "tokensPerSecond": 8.757261172025109,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6962675712,
      "cpuThreads": 4,
      "totalMs": 2260.5430419935146,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1346.6786249991856,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 837.4297920017852,
      "inferenceMs": 837.1525000038673,
      "tokensPerSecond": 9.556203917402197,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6963068928,
      "cpuThreads": 4,
      "totalMs": 2184.108417000971,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1262.5136249989737,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 787.2994170029415,
      "inferenceMs": 787.047125006211,
      "tokensPerSecond": 10.16457559633023,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6963003392,
      "cpuThreads": 1,
      "totalMs": 2049.813042001915,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1262.5136249989737,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 691.472167003667,
      "inferenceMs": 691.1929169946234,
      "tokensPerSecond": 11.574192679497944,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6966034432,
      "cpuThreads": 1,
      "totalMs": 1953.9857920026407,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1262.5136249989737,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 705.4514170013135,
      "inferenceMs": 705.1102089972119,
      "tokensPerSecond": 11.345744109105125,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6966067200,
      "cpuThreads": 1,
      "totalMs": 1967.9650420002872,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1651.5233329992043,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 322
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 798.2207500026561,
      "inferenceMs": 796.5757920028409,
      "tokensPerSecond": 10.042986593762153,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 4323147776,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 516702208,
      "cpuThreads": 4,
      "totalMs": 2449.7440830018604,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1651.5233329992043,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 322
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 581.5407909976784,
      "inferenceMs": 580.1991659973282,
      "tokensPerSecond": 13.788368665178053,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 4323229696,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 529530880,
      "cpuThreads": 4,
      "totalMs": 2233.0641239968827,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1651.5233329992043,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 322
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 330.7444160018349,
      "inferenceMs": 329.1062080024858,
      "tokensPerSecond": 24.308262212846422,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 4323328000,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 539738112,
      "cpuThreads": 4,
      "totalMs": 1982.2677490010392,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1452.7308340038871,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 322
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 201.3932499976363,
      "inferenceMs": 200.22837500437163,
      "tokensPerSecond": 39.95437709478157,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 4323164160,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 508968960,
      "cpuThreads": 4,
      "totalMs": 1654.1240840015234,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1452.7308340038871,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 322
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 193.46025000413647,
      "inferenceMs": 191.34533300530165,
      "tokensPerSecond": 41.80922458024279,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 4323229696,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 519749632,
      "cpuThreads": 4,
      "totalMs": 1646.1910840080236,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1452.7308340038871,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 322
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 193.3153339996352,
      "inferenceMs": 191.3641249993816,
      "tokensPerSecond": 41.80511890630416,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 4323328000,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 530120704,
      "cpuThreads": 4,
      "totalMs": 1646.0461680035223,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1443.0119580065366,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 322
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 208.8688750009169,
      "inferenceMs": 207.3262080011773,
      "tokensPerSecond": 38.586535089449825,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 4323164160,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 511639552,
      "cpuThreads": 4,
      "totalMs": 1651.8808330074535,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1443.0119580065366,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 322
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 197.5936669987277,
      "inferenceMs": 195.48025000403868,
      "tokensPerSecond": 40.92485046358759,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 4323229696,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 522387456,
      "cpuThreads": 4,
      "totalMs": 1640.6056250052643,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
      "torchVersion": "2.10.0",
      "loadMs": 1443.0119580065366,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 322
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 195.39170800271677,
      "inferenceMs": 193.92108300235122,
      "tokensPerSecond": 41.25389501822761,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 4323328000,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 532725760,
      "cpuThreads": 4,
      "totalMs": 1638.4036660092534,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  }
]

Granite 4.0 Dense 350M Base

18 configurations tested
Passed18
Failed0
TestedNov 7, 2026
Runtimetransformers 5.22.1

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
BF16mpsAcceleratedPassedFastmacos · aarch64138 ms / 1051 ms
BF16mpsAcceleratedPassedFastmacos · aarch64141 ms / 1051 ms
BF16mpsAcceleratedPassedFastmacos · aarch64141 ms / 1033 ms
BF16mpsAcceleratedPassedFastmacos · aarch64157 ms / 1051 ms
BF16mpsAcceleratedPassedFastmacos · aarch64160 ms / 1446 ms
BF16mpsAcceleratedPassedFastmacos · aarch64174 ms / 1033 ms
BF16mpsAcceleratedPassedFastmacos · aarch64224 ms / 1033 ms
BF16mpsAcceleratedPassedFastmacos · aarch64270 ms / 1446 ms
BF16mpsAcceleratedPassedFastmacos · aarch64363 ms / 1446 ms
FP32cpuPassedFastmacos · aarch64205 ms / 797 ms
FP32cpuPassedFastmacos · aarch64212 ms / 797 ms
FP32cpuPassedFastmacos · aarch64214 ms / 800 ms
FP32cpuPassedFastmacos · aarch64216 ms / 800 ms
FP32cpuPassedFastmacos · aarch64219 ms / 795 ms
FP32cpuPassedFastmacos · aarch64219 ms / 800 ms
FP32cpuPassedFastmacos · aarch64220 ms / 795 ms
FP32cpuPassedFastmacos · aarch64221 ms / 797 ms
FP32cpuPassedFastmacos · aarch64234 ms / 795 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-07T12:44:37.472396+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 5.22.1
Artifact revision
a50b46cef21c8a86b15f0496cb794487a78a910b
Fixture
{
  "file": "model-samples/desktop/granite4/granite-4.0-350m-base-native-square.json",
  "size": 190,
  "sha1": "931394b01fb0435effd4b4cd4d258fccfbbbd9a0",
  "sha256": "4a5c29b534e7104e5db198844851d6d783f9e861aed9d8841262e5af8182c6d8",
  "url": "https://assets.localai.tools/samples/v1/files/4a5c29b534e7104e5db198844851d6d783f9e861aed9d8841262e5af8182c6d8/granite-4.0-350m-base-native-square.json"
}
Rerun when
Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 1445.6285419946653,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 226
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 271.85124999959953,
      "inferenceMs": 269.9694999973872,
      "tokensPerSecond": 29.632977058806368,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127137280,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 512196608,
      "cpuThreads": 4,
      "totalMs": 1717.4797919942648,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " b"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 1445.6285419946653,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 226
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 161.91129199432908,
      "inferenceMs": 159.7641249973094,
      "tokensPerSecond": 50.073819764823476,
      "inputTokens": 20,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127235584,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 522338304,
      "cpuThreads": 4,
      "totalMs": 1607.5398339889944,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 1445.6285419946653,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 226
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 365.7944160004263,
      "inferenceMs": 363.39537499588914,
      "tokensPerSecond": 22.014589481471795,
      "inputTokens": 15,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127317504,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 534773760,
      "cpuThreads": 4,
      "totalMs": 1811.4229579950916,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 1050.545250000141,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 226
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 159.29970900469925,
      "inferenceMs": 157.4231670019799,
      "tokensPerSecond": 50.81844148072174,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127137280,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 508280832,
      "cpuThreads": 4,
      "totalMs": 1209.8449590048403,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " b"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 1050.545250000141,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 226
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 142.95366599981207,
      "inferenceMs": 140.5202919995645,
      "tokensPerSecond": 56.9312793630175,
      "inputTokens": 20,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127251968,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 518750208,
      "cpuThreads": 4,
      "totalMs": 1193.4989159999532,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 1050.545250000141,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 226
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 140.79087500431342,
      "inferenceMs": 138.1451669949456,
      "tokensPerSecond": 57.91009684973417,
      "inputTokens": 15,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127333888,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 528891904,
      "cpuThreads": 4,
      "totalMs": 1191.3361250044545,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 1033.4577500034356,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 226
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 176.33650000061607,
      "inferenceMs": 174.40383299981477,
      "tokensPerSecond": 45.8705514804167,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127137280,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 504872960,
      "cpuThreads": 4,
      "totalMs": 1209.7942500040517,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " b"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 1033.4577500034356,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 226
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 143.06125000439351,
      "inferenceMs": 140.84675000049174,
      "tokensPerSecond": 56.79932266787888,
      "inputTokens": 20,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127251968,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 515440640,
      "cpuThreads": 4,
      "totalMs": 1176.5190000078292,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 1033.4577500034356,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 226
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 226.2833330023568,
      "inferenceMs": 224.37091599567793,
      "tokensPerSecond": 35.65524508601687,
      "inputTokens": 15,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127333888,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 525664256,
      "cpuThreads": 4,
      "totalMs": 1259.7410830057925,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 795.2704999988782,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 226
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 220.78666699962923,
      "inferenceMs": 220.4687909979839,
      "tokensPerSecond": 36.28631500987891,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1827995648,
      "cpuThreads": 4,
      "totalMs": 1016.0571669985075,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " b"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 795.2704999988782,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 226
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 234.10525000508642,
      "inferenceMs": 233.88374999922235,
      "tokensPerSecond": 34.205027070185935,
      "inputTokens": 20,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1829732352,
      "cpuThreads": 4,
      "totalMs": 1029.3757500039646,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 795.2704999988782,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 226
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 219.4556249960442,
      "inferenceMs": 219.2440409999108,
      "tokensPerSecond": 36.48901910179285,
      "inputTokens": 15,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1834172416,
      "cpuThreads": 4,
      "totalMs": 1014.7261249949224,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 796.7337919981219,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 226
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 221.40679199947044,
      "inferenceMs": 221.13987500051735,
      "tokensPerSecond": 36.176198435407834,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1830043648,
      "cpuThreads": 4,
      "totalMs": 1018.1405839975923,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " b"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 796.7337919981219,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 226
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 211.8561250026687,
      "inferenceMs": 211.6536250032368,
      "tokensPerSecond": 37.79760445812188,
      "inputTokens": 20,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1832009728,
      "cpuThreads": 4,
      "totalMs": 1008.5899170007906,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 796.7337919981219,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 226
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 205.2512080044835,
      "inferenceMs": 205.05916700494708,
      "tokensPerSecond": 39.01313029232679,
      "inputTokens": 15,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1836515328,
      "cpuThreads": 4,
      "totalMs": 1001.9850000026054,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 799.9750829985715,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 226
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 219.75579200079665,
      "inferenceMs": 219.4953330035787,
      "tokensPerSecond": 36.447244187508836,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1826062336,
      "cpuThreads": 4,
      "totalMs": 1019.7308749993681,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " b"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 799.9750829985715,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 226
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 216.37612499762326,
      "inferenceMs": 216.16479100339347,
      "tokensPerSecond": 37.008802233081575,
      "inputTokens": 20,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1827700736,
      "cpuThreads": 4,
      "totalMs": 1016.3512079961947,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
      "torchVersion": "2.10.0",
      "loadMs": 799.9750829985715,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 226
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 213.9065000010305,
      "inferenceMs": 213.70541700161994,
      "tokensPerSecond": 37.43470854526518,
      "inputTokens": 15,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1832271872,
      "cpuThreads": 4,
      "totalMs": 1013.881582999602,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  }
]

Granite 4.0 Hybrid 1B Base

15 configurations tested
Passed9
Failed6
TestedNov 7, 2026
Runtimetransformers 5.22.1

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedFastmacos · aarch64375 ms / 1492 ms
FP16mpsAcceleratedPassedFastmacos · aarch64378 ms / 1506 ms
FP16mpsAcceleratedPassedFastmacos · aarch64379 ms / 1492 ms
FP16mpsAcceleratedPassedFastmacos · aarch64381 ms / 1492 ms
FP16mpsAcceleratedPassedFastmacos · aarch64394 ms / 1506 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64409 ms / 1506 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64438 ms / 1873 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64493 ms / 1873 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64546 ms / 1873 ms
FP32cpuFailed—macos · aarch6417764 ms / 3538 ms
FP32cpuFailed—macos · aarch6439103 ms / 3538 ms
FP32cpuFailed—macos · aarch6454682 ms / 3538 ms
FP32cpuFailed—macos · aarch647163 ms / 4246 ms
FP32cpuFailed—macos · aarch643894 ms / 4246 ms
FP32cpuFailed—macos · aarch643768 ms / 4246 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-07T12:40:38.588591+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 5.22.1
Artifact revision
8d34631309affca6f6f8350684ddcb8994b0dc31
Fixture
{
  "file": "model-samples/desktop/granite4/granite-4.0-h-1b-base-native-geography.json",
  "size": 152,
  "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef",
  "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21",
  "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-h-1b-base-native-geography.json"
}
Rerun when
Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 1873.1615420037997,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 547.3154579958646,
      "inferenceMs": 545.8465839983546,
      "tokensPerSecond": 14.656132756936177,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 3258007552,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 1603469312,
      "cpuThreads": 4,
      "totalMs": 2420.4769999996643,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 1873.1615420037997,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 494.6955839986913,
      "inferenceMs": 493.1372920036665,
      "tokensPerSecond": 16.22266279537529,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 3258138624,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 1622294528,
      "cpuThreads": 4,
      "totalMs": 2367.857126002491,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 1873.1615420037997,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 440.1028749998659,
      "inferenceMs": 438.37654199887766,
      "tokensPerSecond": 18.24915166199856,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 3258286080,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 1638350848,
      "cpuThreads": 4,
      "totalMs": 2313.2644170036656,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 1506.4057080016937,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 410.5535409980803,
      "inferenceMs": 408.742166000593,
      "tokensPerSecond": 19.57224056005123,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 3258023936,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 1599504384,
      "cpuThreads": 4,
      "totalMs": 1916.959248999774,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 1506.4057080016937,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 396.11779199913144,
      "inferenceMs": 394.09499999601394,
      "tokensPerSecond": 20.299673936692717,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 3258138624,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 1617215488,
      "cpuThreads": 4,
      "totalMs": 1902.5235000008252,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 1506.4057080016937,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 380.40933400043286,
      "inferenceMs": 378.2888749992708,
      "tokensPerSecond": 21.147859555783715,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 3258286080,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 1633583104,
      "cpuThreads": 4,
      "totalMs": 1886.8150420021266,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 1492.281416998594,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 382.5119169996469,
      "inferenceMs": 381.00200000189943,
      "tokensPerSecond": 20.997265106115236,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 3258023936,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 1601404928,
      "cpuThreads": 4,
      "totalMs": 1874.793333998241,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 1492.281416998594,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 380.5726249993313,
      "inferenceMs": 378.9085829994292,
      "tokensPerSecond": 21.113272063335792,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 3258138624,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 1619640320,
      "cpuThreads": 4,
      "totalMs": 1872.8540419979254,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 1492.281416998594,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 377.0788330002688,
      "inferenceMs": 375.4507920020842,
      "tokensPerSecond": 21.30771906843012,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 3258286080,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 1637662720,
      "cpuThreads": 4,
      "totalMs": 1869.3602499988629,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": false,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 3537.7601660002256,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 466
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 17764.568624996173,
      "inferenceMs": 17764.357082996867,
      "tokensPerSecond": 0.4503399679832596,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6441222144,
      "cpuThreads": 4,
      "totalMs": 21302.3287909964,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 3537.7601660002256,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 466
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 39103.663957997924,
      "inferenceMs": 39103.35812500125,
      "tokensPerSecond": 0.20458600958072432,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6444564480,
      "cpuThreads": 4,
      "totalMs": 42641.42412399815,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 3537.7601660002256,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 466
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 54682.723291,
      "inferenceMs": 54682.426084000326,
      "tokensPerSecond": 0.1462992879597334,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6449283072,
      "cpuThreads": 4,
      "totalMs": 58220.48345700023,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": false,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 4246.428250000463,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 466
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 7163.7770000015735,
      "inferenceMs": 7163.414041999204,
      "tokensPerSecond": 1.1167859282034907,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6141591552,
      "cpuThreads": 1,
      "totalMs": 11410.205250002036,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " Celsius",
      "fixturePassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 4246.428250000463,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 466
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 3894.8861669996404,
      "inferenceMs": 3894.0525840007467,
      "tokensPerSecond": 2.0544149898923054,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6152323072,
      "cpuThreads": 1,
      "totalMs": 8141.314417000103,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    },
    "failure": "This fixture passed but the backend did not pass the complete repeated profile."
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " x",
      "fixturePassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
      "torchVersion": "2.10.0",
      "loadMs": 4246.428250000463,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 466
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 3768.907333003881,
      "inferenceMs": 3768.443249995471,
      "tokensPerSecond": 2.122892523327667,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 6159450112,
      "cpuThreads": 1,
      "totalMs": 8015.335583004344,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    },
    "failure": "This fixture passed but the backend did not pass the complete repeated profile."
  }
]

Granite 4.0 Hybrid 350M Base

15 configurations tested
Passed9
Failed6
TestedNov 7, 2026
Runtimetransformers 5.22.1

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedFastmacos · aarch64374 ms / 1374 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64419 ms / 1374 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64429 ms / 1252 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64465 ms / 1252 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64517 ms / 1252 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64519 ms / 1374 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64559 ms / 1304 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64694 ms / 1304 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64697 ms / 1304 ms
FP32cpuFailed—macos · aarch646757 ms / 1800 ms
FP32cpuFailed—macos · aarch646840 ms / 1800 ms
FP32cpuFailed—macos · aarch646794 ms / 1800 ms
FP32cpuFailed—macos · aarch641705 ms / 1475 ms
FP32cpuFailed—macos · aarch641715 ms / 1475 ms
FP32cpuFailed—macos · aarch641705 ms / 1475 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-07T12:28:48.153999+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 5.22.1
Artifact revision
dc555b6939c863bb96034d1eae7601da36bd42e4
Fixture
{
  "file": "model-samples/desktop/granite4/granite-4.0-h-350m-base-native-geography.json",
  "size": 152,
  "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef",
  "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21",
  "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-h-350m-base-native-geography.json"
}
Rerun when
Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1304.3914590016357,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 370
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 697.7576670033159,
      "inferenceMs": 696.7547079984797,
      "tokensPerSecond": 11.481802574368045,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127251968,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 834699264,
      "cpuThreads": 4,
      "totalMs": 2002.1491260049515,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1304.3914590016357,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 370
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 695.3992080016178,
      "inferenceMs": 693.9490000004298,
      "tokensPerSecond": 11.528224696620422,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127366656,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 852197376,
      "cpuThreads": 4,
      "totalMs": 1999.7906670032535,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1304.3914590016357,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 370
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 560.810875002062,
      "inferenceMs": 559.3551249985467,
      "tokensPerSecond": 14.302184144680512,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127497728,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 869105664,
      "cpuThreads": 4,
      "totalMs": 1865.2023340036976,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1251.805959000194,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 370
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 466.8252499977825,
      "inferenceMs": 465.2519589944859,
      "tokensPerSecond": 17.19498401960477,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127268352,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 831782912,
      "cpuThreads": 4,
      "totalMs": 1718.6312089979765,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1251.805959000194,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 370
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 518.8388750029844,
      "inferenceMs": 517.4960419972194,
      "tokensPerSecond": 15.459055433786265,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127366656,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 849592320,
      "cpuThreads": 4,
      "totalMs": 1770.6448340031784,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1251.805959000194,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 370
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 431.575791000796,
      "inferenceMs": 429.3407910008682,
      "tokensPerSecond": 18.63321670729354,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127497728,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 866091008,
      "cpuThreads": 4,
      "totalMs": 1683.38175000099,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1374.3903750000754,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 370
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 519.7690419954597,
      "inferenceMs": 518.6717499964288,
      "tokensPerSecond": 15.424013357301765,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127251968,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 830521344,
      "cpuThreads": 4,
      "totalMs": 1894.1594169955351,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1374.3903750000754,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 370
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 420.21004200069,
      "inferenceMs": 419.1607080065296,
      "tokensPerSecond": 19.085758390968692,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127366656,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 847314944,
      "cpuThreads": 4,
      "totalMs": 1794.6004170007654,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1374.3903750000754,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 370
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 374.6143330063205,
      "inferenceMs": 373.58783400122775,
      "tokensPerSecond": 21.413973560963736,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 1127497728,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 863617024,
      "cpuThreads": 4,
      "totalMs": 1749.004708006396,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": false,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1799.7390419986914,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 370
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 6757.583833998069,
      "inferenceMs": 6757.353875000263,
      "tokensPerSecond": 1.1838953750220296,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1909538816,
      "cpuThreads": 4,
      "totalMs": 8557.32287599676,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1799.7390419986914,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 370
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 6839.964625003631,
      "inferenceMs": 6839.769833000901,
      "tokensPerSecond": 1.1696300014952485,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1909800960,
      "cpuThreads": 4,
      "totalMs": 8639.703667002323,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1799.7390419986914,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 370
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 6794.528875005199,
      "inferenceMs": 6794.299333996605,
      "tokensPerSecond": 1.177457690150688,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1911603200,
      "cpuThreads": 4,
      "totalMs": 8594.26791700389,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": false,
      "expected": " France",
      "fixturePassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1475.008749999688,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 370
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1705.6864999976824,
      "inferenceMs": 1705.4741669999203,
      "tokensPerSecond": 4.690777588307132,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1903198208,
      "cpuThreads": 1,
      "totalMs": 3180.6952499973704,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    },
    "failure": "This fixture passed but the backend did not pass the complete repeated profile."
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " Celsius",
      "fixturePassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1475.008749999688,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 370
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1715.0565419942723,
      "inferenceMs": 1714.8642910033232,
      "tokensPerSecond": 4.665092183661604,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1907638272,
      "cpuThreads": 1,
      "totalMs": 3190.0652919939603,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    },
    "failure": "This fixture passed but the backend did not pass the complete repeated profile."
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " x",
      "fixturePassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
      "torchVersion": "2.10.0",
      "loadMs": 1475.008749999688,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 370
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1705.6674159975955,
      "inferenceMs": 1705.4481669983943,
      "tokensPerSecond": 4.690849100433278,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 1907933184,
      "cpuThreads": 1,
      "totalMs": 3180.6761659972835,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    },
    "failure": "This fixture passed but the backend did not pass the complete repeated profile."
  }
]

Granite 4.0 Hybrid Micro Base

15 configurations tested
Passed9
Failed6
TestedNov 7, 2026
Runtimetransformers 5.22.1

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedModeratemacos · aarch64460 ms / 2028 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64460 ms / 2635 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64462 ms / 2028 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64469 ms / 2028 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64475 ms / 2635 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64480 ms / 2635 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64484 ms / 2057 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64487 ms / 2057 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64498 ms / 2057 ms
FP32cpuFailed—macos · aarch6423114 ms / 6096 ms
FP32cpuFailed—macos · aarch6430796 ms / 6096 ms
FP32cpuFailed—macos · aarch6427770 ms / 6096 ms
FP32cpuFailed—macos · aarch645637 ms / 7172 ms
FP32cpuFailed—macos · aarch645487 ms / 7172 ms
FP32cpuFailed—macos · aarch645643 ms / 7172 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-07T12:47:48.570317+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 5.22.1
Artifact revision
372ede0bc484284e08ce574e773c96cd2b89b367
Fixture
{
  "file": "model-samples/desktop/granite4/granite-4.0-h-micro-base-native-geography.json",
  "size": 152,
  "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef",
  "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21",
  "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-h-micro-base-native-geography.json"
}
Rerun when
Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 2634.6227920003003,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 476.8214590003481,
      "inferenceMs": 475.352458000998,
      "tokensPerSecond": 16.829617403562903,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7553040384,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2942648320,
      "cpuThreads": 4,
      "totalMs": 3111.4442510006484,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 2634.6227920003003,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 461.8705000029877,
      "inferenceMs": 460.3202500002226,
      "tokensPerSecond": 17.379205020843926,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7553204224,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2960900096,
      "cpuThreads": 4,
      "totalMs": 3096.493292003288,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 2634.6227920003003,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 481.88800000207266,
      "inferenceMs": 480.14949999924283,
      "tokensPerSecond": 16.661477310738874,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7554465792,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2977841152,
      "cpuThreads": 4,
      "totalMs": 3116.510792002373,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 2028.0584579959395,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 470.57316699647345,
      "inferenceMs": 468.704582999635,
      "tokensPerSecond": 17.068320409417097,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7553040384,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2938142720,
      "cpuThreads": 4,
      "totalMs": 2498.631624992413,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 2028.0584579959395,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 461.40058300079545,
      "inferenceMs": 459.86562500183936,
      "tokensPerSecond": 17.396386172521595,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7553204224,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2956017664,
      "cpuThreads": 4,
      "totalMs": 2489.459040996735,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 2028.0584579959395,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 463.87520900316304,
      "inferenceMs": 462.3458749993006,
      "tokensPerSecond": 17.303063426297687,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7554465792,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2972598272,
      "cpuThreads": 4,
      "totalMs": 2491.9336669991026,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 2056.556499999715,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 488.31754199636634,
      "inferenceMs": 487.294833001215,
      "tokensPerSecond": 16.4171656627848,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7553040384,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2940272640,
      "cpuThreads": 4,
      "totalMs": 2544.8740419960814,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 2056.556499999715,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 499.15637499361765,
      "inferenceMs": 497.5939579962869,
      "tokensPerSecond": 16.07736563404915,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7553204224,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2958376960,
      "cpuThreads": 4,
      "totalMs": 2555.7128749933327,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 2056.556499999715,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 466
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 485.33845800557174,
      "inferenceMs": 483.688667001843,
      "tokensPerSecond": 16.539564694761637,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7554465792,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2975121408,
      "cpuThreads": 4,
      "totalMs": 2541.8949580052868,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": false,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 6095.792624997557,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 466
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 23114.043624998885,
      "inferenceMs": 23113.81395799981,
      "tokensPerSecond": 0.3461133681588347,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 13419200512,
      "cpuThreads": 4,
      "totalMs": 29209.836249996442,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 6095.792624997557,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 466
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 30796.594999999797,
      "inferenceMs": 30796.317583000928,
      "tokensPerSecond": 0.2597713177375425,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 12974817280,
      "cpuThreads": 4,
      "totalMs": 36892.387624997355,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 6095.792624997557,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 466
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 27771.326832997147,
      "inferenceMs": 27770.42245800112,
      "tokensPerSecond": 0.2880762801537816,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 13113786368,
      "cpuThreads": 4,
      "totalMs": 33867.119457994704,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": false,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 7172.052750000148,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 466
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 5637.365375005174,
      "inferenceMs": 5637.127166999562,
      "tokensPerSecond": 1.4191625916890764,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 13412777984,
      "cpuThreads": 1,
      "totalMs": 12809.418125005323,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 7172.052750000148,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 466
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 5486.838208998961,
      "inferenceMs": 5486.58629199781,
      "tokensPerSecond": 1.4581015542702767,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 13417070592,
      "cpuThreads": 1,
      "totalMs": 12658.890958999109,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
      "torchVersion": "2.10.0",
      "loadMs": 7172.052750000148,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 466
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 5643.402708003123,
      "inferenceMs": 5643.151708005462,
      "tokensPerSecond": 1.4176475157758168,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 13417594880,
      "cpuThreads": 1,
      "totalMs": 12815.45545800327,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  }
]

Granite 4.0 Hybrid Small Base

21 configurations tested
Passed21
Failed0
TestedNov 7, 2026
Runtimellama.cpp 0.3.36

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
Q4mpsAcceleratedPassedFastmacos · aarch64379 ms / 1452 ms
Q4mpsAcceleratedPassedFastmacos · aarch64381 ms / 1365 ms
Q4mpsAcceleratedPassedFastmacos · aarch64382 ms / 1373 ms
Q4mpsAcceleratedPassedModeratemacos · aarch64408 ms / 1365 ms
Q4mpsAcceleratedPassedModeratemacos · aarch64415 ms / 1373 ms
Q4mpsAcceleratedPassedModeratemacos · aarch64415 ms / 1452 ms
Q4mpsAcceleratedPassedModeratemacos · aarch64418 ms / 1373 ms
Q4mpsAcceleratedPassedModeratemacos · aarch64421 ms / 1365 ms
Q4mpsAcceleratedPassedModeratemacos · aarch64460 ms / 1452 ms
Q4cpuPassedModeratemacos · aarch64738 ms / 18978 ms
Q4cpuPassedModeratemacos · aarch64756 ms / 19114 ms
Q4cpuPassedModeratemacos · aarch64764 ms / 18997 ms
Q4cpuPassedModeratemacos · aarch64804 ms / 18978 ms
Q4cpuPassedModeratemacos · aarch64809 ms / 19114 ms
Q4cpuPassedModeratemacos · aarch64831 ms / 18997 ms
Q4cpuPassedModeratemacos · aarch64904 ms / 19114 ms
Q4cpuPassedModeratemacos · aarch64922 ms / 18978 ms
Q4cpuPassedModeratemacos · aarch64956 ms / 18997 ms
Q4cpuPassedModeratemacos · aarch642276 ms / 19138 ms
Q4cpuPassedModeratemacos · aarch642580 ms / 19138 ms
Q4cpuPassedModeratemacos · aarch643015 ms / 19138 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-07T13:01:05.605185+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
llama.cpp 0.3.36
Artifact revision
d613b92b3764b8142d1a4f64017fb457505dbc66
Fixture
{
  "file": "model-samples/desktop/granite4/granite-4.0-h-small-base-native-geography.json",
  "size": 152,
  "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef",
  "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21",
  "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-h-small-base-native-geography.json"
}
Rerun when
Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 1452.0784169944818,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   321.56 MiB\nload_tensors:  MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =   128.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =   147.48 MiB\nsched_reserve:       MTL0 compute buffer size =   282.28 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 459.63704199675703,
      "durationMs": 459.7870409997995,
      "tokensPerSecond": 17.405037603684786,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 19938377728,
      "cpuThreads": 4,
      "totalMs": 1911.8654579942813,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 1452.0784169944818,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   321.56 MiB\nload_tensors:  MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =   128.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =   147.48 MiB\nsched_reserve:       MTL0 compute buffer size =   282.28 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 378.67499999993015,
      "durationMs": 378.8076250057202,
      "tokensPerSecond": 21.126295636103453,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 19940065280,
      "cpuThreads": 4,
      "totalMs": 1830.886042000202,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 1452.0784169944818,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   321.56 MiB\nload_tensors:  MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =   128.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =   147.48 MiB\nsched_reserve:       MTL0 compute buffer size =   282.28 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 414.9059170013061,
      "durationMs": 415.087458997732,
      "tokensPerSecond": 19.28147966126696,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 19941605376,
      "cpuThreads": 4,
      "totalMs": 1867.1658759922138,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 1364.768790997914,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   321.56 MiB\nload_tensors:  MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =   128.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =   147.48 MiB\nsched_reserve:       MTL0 compute buffer size =   282.28 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 407.76229200128,
      "durationMs": 408.06637499917997,
      "tokensPerSecond": 19.619273672257286,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 19933691904,
      "cpuThreads": 4,
      "totalMs": 1772.835165997094,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 1364.768790997914,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   321.56 MiB\nload_tensors:  MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =   128.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =   147.48 MiB\nsched_reserve:       MTL0 compute buffer size =   282.28 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 381.23891700524837,
      "durationMs": 381.40541699976893,
      "tokensPerSecond": 20.98421657170395,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 19935248384,
      "cpuThreads": 4,
      "totalMs": 1746.174207997683,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 1364.768790997914,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   321.56 MiB\nload_tensors:  MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =   128.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =   147.48 MiB\nsched_reserve:       MTL0 compute buffer size =   282.28 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 421.3492910057539,
      "durationMs": 421.6057079975144,
      "tokensPerSecond": 18.9866226685801,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 19936837632,
      "cpuThreads": 4,
      "totalMs": 1786.3744989954284,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 1372.6486250016023,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   321.56 MiB\nload_tensors:  MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =   128.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =   147.48 MiB\nsched_reserve:       MTL0 compute buffer size =   282.28 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 414.8267909986316,
      "durationMs": 415.06904199923156,
      "tokensPerSecond": 19.285157500896293,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 19935756288,
      "cpuThreads": 4,
      "totalMs": 1787.717667000834,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 1372.6486250016023,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   321.56 MiB\nload_tensors:  MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =   128.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =   147.48 MiB\nsched_reserve:       MTL0 compute buffer size =   282.28 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 381.8154170003254,
      "durationMs": 382.1792919989093,
      "tokensPerSecond": 20.952532673643145,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 19937378304,
      "cpuThreads": 4,
      "totalMs": 1754.8279170005117,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 1372.6486250016023,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   321.56 MiB\nload_tensors:  MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =   128.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =   147.48 MiB\nsched_reserve:       MTL0 compute buffer size =   282.28 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 418.05729199404595,
      "durationMs": 418.2641669976874,
      "tokensPerSecond": 19.136133140607765,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 19939016704,
      "cpuThreads": 4,
      "totalMs": 1790.9127919992898,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 19113.816709003004,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors:   CPU_REPACK model buffer size = 18517.69 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 755.6653749998077,
      "durationMs": 756.0884159975103,
      "tokensPerSecond": 10.586696525564687,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 39355023360,
      "cpuThreads": 4,
      "totalMs": 19869.905125000514,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 19113.816709003004,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors:   CPU_REPACK model buffer size = 18517.69 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 808.9200410031481,
      "durationMs": 809.127417000127,
      "tokensPerSecond": 9.889729014599684,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 39359135744,
      "cpuThreads": 4,
      "totalMs": 19922.94412600313,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 19113.816709003004,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors:   CPU_REPACK model buffer size = 18517.69 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 904.219625001133,
      "durationMs": 904.4215829999303,
      "tokensPerSecond": 8.847408061940678,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 39364132864,
      "cpuThreads": 4,
      "totalMs": 20018.238292002934,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 18977.763041002618,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors:   CPU_REPACK model buffer size = 18517.69 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 737.6848749991041,
      "durationMs": 737.9867909985478,
      "tokensPerSecond": 10.844739089993835,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 39353843712,
      "cpuThreads": 4,
      "totalMs": 19715.749832001165,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 18977.763041002618,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors:   CPU_REPACK model buffer size = 18517.69 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 803.7167500006035,
      "durationMs": 803.8210840022657,
      "tokensPerSecond": 9.953755474169219,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 39356825600,
      "cpuThreads": 4,
      "totalMs": 19781.584125004883,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 18977.763041002618,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors:   CPU_REPACK model buffer size = 18517.69 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 922.4196249997476,
      "durationMs": 922.5294590069097,
      "tokensPerSecond": 8.672842362826126,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 39361822720,
      "cpuThreads": 4,
      "totalMs": 19900.292500009527,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 18997.33595900034,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors:   CPU_REPACK model buffer size = 18517.69 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 763.7237500021001,
      "durationMs": 763.8918750017183,
      "tokensPerSecond": 10.474991775465934,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 39352565760,
      "cpuThreads": 4,
      "totalMs": 19761.227834002057,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 18997.33595900034,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors:   CPU_REPACK model buffer size = 18517.69 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 830.6357090041274,
      "durationMs": 830.7312500037369,
      "tokensPerSecond": 9.631177558681442,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 39356645376,
      "cpuThreads": 4,
      "totalMs": 19828.067209004075,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 18997.33595900034,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors:   CPU_REPACK model buffer size = 18517.69 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 955.6527500026277,
      "durationMs": 955.8113329985645,
      "tokensPerSecond": 8.371241541426008,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 39361429504,
      "cpuThreads": 4,
      "totalMs": 19953.147291998903,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 19138.133375003235,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors:   CPU_REPACK model buffer size = 18517.69 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 2276.01987500384,
      "durationMs": 2276.3410420011496,
      "tokensPerSecond": 3.514907794900738,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 39353434112,
      "cpuThreads": 1,
      "totalMs": 21414.474417004385,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 19138.133375003235,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors:   CPU_REPACK model buffer size = 18517.69 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 2579.533416996128,
      "durationMs": 2580.1095420029014,
      "tokensPerSecond": 3.101336058408585,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 39357415424,
      "cpuThreads": 1,
      "totalMs": 21718.242917006137,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 19138.133375003235,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors:   CPU_REPACK model buffer size = 18517.69 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 3014.7489169976325,
      "durationMs": 3014.860041999782,
      "tokensPerSecond": 2.653620656398525,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 39362150400,
      "cpuThreads": 1,
      "totalMs": 22152.993417003017,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  }
]

Granite 4.0 Hybrid Tiny Base

21 configurations tested
Passed19
Failed2
TestedNov 7, 2026
Runtimellama.cpp 0.3.36

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
Q4mpsAcceleratedPassedFastmacos · aarch64114 ms / 385 ms
Q4mpsAcceleratedPassedFastmacos · aarch64115 ms / 385 ms
Q4mpsAcceleratedPassedFastmacos · aarch64119 ms / 386 ms
Q4mpsAcceleratedPassedFastmacos · aarch64119 ms / 386 ms
Q4mpsAcceleratedPassedFastmacos · aarch64122 ms / 385 ms
Q4mpsAcceleratedPassedFastmacos · aarch64124 ms / 451 ms
Q4mpsAcceleratedPassedFastmacos · aarch64125 ms / 386 ms
Q4mpsAcceleratedPassedFastmacos · aarch64126 ms / 451 ms
Q4mpsAcceleratedPassedFastmacos · aarch64268 ms / 451 ms
Q4cpuPassedFastmacos · aarch64206 ms / 4300 ms
Q4cpuPassedFastmacos · aarch64210 ms / 4300 ms
Q4cpuPassedFastmacos · aarch64222 ms / 4300 ms
Q4cpuPassedModeratemacos · aarch64569 ms / 6245 ms
Q4cpuPassedModeratemacos · aarch64658 ms / 6245 ms
Q4cpuPassedModeratemacos · aarch64779 ms / 6857 ms
Q4cpuPassedModeratemacos · aarch64810 ms / 6857 ms
Q4cpuPassedModeratemacos · aarch641210 ms / 6857 ms
Q4cpuPassedModeratemacos · aarch642132 ms / 6245 ms
Q4cpuPassedModeratemacos · aarch643805 ms / 120087 ms
Q4cpuFailed—macos · aarch6412897 ms / 120087 ms
Q4cpuFailed—macos · aarch644405 ms / 120087 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-07T12:53:09.537151+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
llama.cpp 0.3.36
Artifact revision
27f75056a8aabab6377df1d9dd4f37c9b1fbe14e
Fixture
{
  "file": "model-samples/desktop/granite4/granite-4.0-h-tiny-base-native-geography.json",
  "size": 152,
  "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef",
  "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21",
  "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-h-tiny-base-native-geography.json"
}
Rerun when
Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 450.72349999827566,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   120.59 MiB\nload_tensors:  MTL0_Mapped model buffer size =  4031.55 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =    64.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =    55.37 MiB\nsched_reserve:       MTL0 compute buffer size =   123.63 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 268.0246250019991,
      "durationMs": 268.3380840026075,
      "tokensPerSecond": 29.848003704660837,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 4526260224,
      "cpuThreads": 4,
      "totalMs": 719.0615840008832,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 450.72349999827566,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   120.59 MiB\nload_tensors:  MTL0_Mapped model buffer size =  4031.55 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =    64.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =    55.37 MiB\nsched_reserve:       MTL0 compute buffer size =   123.63 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 125.9983329946408,
      "durationMs": 126.19366600119974,
      "tokensPerSecond": 63.49290351595581,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 4527570944,
      "cpuThreads": 4,
      "totalMs": 576.9171659994754,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 450.72349999827566,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   120.59 MiB\nload_tensors:  MTL0_Mapped model buffer size =  4031.55 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =    64.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =    55.37 MiB\nsched_reserve:       MTL0 compute buffer size =   123.63 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 123.9338330051396,
      "durationMs": 124.15400000463706,
      "tokensPerSecond": 64.55057352795855,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 4528963584,
      "cpuThreads": 4,
      "totalMs": 574.8775000029127,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 385.37112500489457,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   120.59 MiB\nload_tensors:  MTL0_Mapped model buffer size =  4031.55 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =    64.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =    55.37 MiB\nsched_reserve:       MTL0 compute buffer size =   123.63 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 114.76595799467759,
      "durationMs": 114.93583300034516,
      "tokensPerSecond": 69.70708161013225,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 4525637632,
      "cpuThreads": 4,
      "totalMs": 500.3069580052397,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 385.37112500489457,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   120.59 MiB\nload_tensors:  MTL0_Mapped model buffer size =  4031.55 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =    64.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =    55.37 MiB\nsched_reserve:       MTL0 compute buffer size =   123.63 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 114.07291599607561,
      "durationMs": 114.18383399723098,
      "tokensPerSecond": 70.13058209430905,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 4526866432,
      "cpuThreads": 4,
      "totalMs": 499.55495900212554,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 385.37112500489457,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   120.59 MiB\nload_tensors:  MTL0_Mapped model buffer size =  4031.55 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =    64.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =    55.37 MiB\nsched_reserve:       MTL0 compute buffer size =   123.63 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 121.5497500015772,
      "durationMs": 121.70024999795714,
      "tokensPerSecond": 65.81667177346061,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 4528324608,
      "cpuThreads": 4,
      "totalMs": 507.0713750028517,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 386.3765839996631,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   120.59 MiB\nload_tensors:  MTL0_Mapped model buffer size =  4031.55 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =    64.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =    55.37 MiB\nsched_reserve:       MTL0 compute buffer size =   123.63 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 118.87858400586993,
      "durationMs": 119.17083300068043,
      "tokensPerSecond": 67.29555257492788,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 4528193536,
      "cpuThreads": 4,
      "totalMs": 505.54741700034356,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 386.3765839996631,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   120.59 MiB\nload_tensors:  MTL0_Mapped model buffer size =  4031.55 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =    64.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =    55.37 MiB\nsched_reserve:       MTL0 compute buffer size =   123.63 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 119.46229099703487,
      "durationMs": 119.69924999721115,
      "tokensPerSecond": 66.96673848485432,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 4529717248,
      "cpuThreads": 4,
      "totalMs": 506.0758339968743,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "mps",
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "loadMs": 386.3765839996631,
      "acceleration": "partial",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer   0 assigned to device MTL0, is_swa = 0\nload_tensors: layer   1 assigned to device MTL0, is_swa = 0\nload_tensors: layer   2 assigned to device MTL0, is_swa = 0\nload_tensors: layer   3 assigned to device MTL0, is_swa = 0\nload_tensors: layer   4 assigned to device MTL0, is_swa = 0\nload_tensors: layer   5 assigned to device MTL0, is_swa = 0\nload_tensors: layer   6 assigned to device MTL0, is_swa = 0\nload_tensors: layer   7 assigned to device MTL0, is_swa = 0\nload_tensors: layer   8 assigned to device MTL0, is_swa = 0\nload_tensors: layer   9 assigned to device MTL0, is_swa = 0\nload_tensors: layer  10 assigned to device MTL0, is_swa = 0\nload_tensors: layer  11 assigned to device MTL0, is_swa = 0\nload_tensors: layer  12 assigned to device MTL0, is_swa = 0\nload_tensors: layer  13 assigned to device MTL0, is_swa = 0\nload_tensors: layer  14 assigned to device MTL0, is_swa = 0\nload_tensors: layer  15 assigned to device MTL0, is_swa = 0\nload_tensors: layer  16 assigned to device MTL0, is_swa = 0\nload_tensors: layer  17 assigned to device MTL0, is_swa = 0\nload_tensors: layer  18 assigned to device MTL0, is_swa = 0\nload_tensors: layer  19 assigned to device MTL0, is_swa = 0\nload_tensors: layer  20 assigned to device MTL0, is_swa = 0\nload_tensors: layer  21 assigned to device MTL0, is_swa = 0\nload_tensors: layer  22 assigned to device MTL0, is_swa = 0\nload_tensors: layer  23 assigned to device MTL0, is_swa = 0\nload_tensors: layer  24 assigned to device MTL0, is_swa = 0\nload_tensors: layer  25 assigned to device MTL0, is_swa = 0\nload_tensors: layer  26 assigned to device MTL0, is_swa = 0\nload_tensors: layer  27 assigned to device MTL0, is_swa = 0\nload_tensors: layer  28 assigned to device MTL0, is_swa = 0\nload_tensors: layer  29 assigned to device MTL0, is_swa = 0\nload_tensors: layer  30 assigned to device MTL0, is_swa = 0\nload_tensors: layer  31 assigned to device MTL0, is_swa = 0\nload_tensors: layer  32 assigned to device MTL0, is_swa = 0\nload_tensors: layer  33 assigned to device MTL0, is_swa = 0\nload_tensors: layer  34 assigned to device MTL0, is_swa = 0\nload_tensors: layer  35 assigned to device MTL0, is_swa = 0\nload_tensors: layer  36 assigned to device MTL0, is_swa = 0\nload_tensors: layer  37 assigned to device MTL0, is_swa = 0\nload_tensors: layer  38 assigned to device MTL0, is_swa = 0\nload_tensors: layer  39 assigned to device MTL0, is_swa = 0\nload_tensors: layer  40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =   120.59 MiB\nload_tensors:  MTL0_Mapped model buffer size =  4031.55 MiB\nllama_kv_cache: layer   5: dev = MTL0\nllama_kv_cache: layer  15: dev = MTL0\nllama_kv_cache: layer  25: dev = MTL0\nllama_kv_cache: layer  35: dev = MTL0\nllama_kv_cache:       MTL0 KV buffer size =    64.00 MiB\nllama_memory_recurrent, layer   0: dev = MTL0\nllama_memory_recurrent, layer   1: dev = MTL0\nllama_memory_recurrent, layer   2: dev = MTL0\nllama_memory_recurrent, layer   3: dev = MTL0\nllama_memory_recurrent, layer   4: dev = MTL0\nllama_memory_recurrent, layer   6: dev = MTL0\nllama_memory_recurrent, layer   7: dev = MTL0\nllama_memory_recurrent, layer   8: dev = MTL0\nllama_memory_recurrent, layer   9: dev = MTL0\nllama_memory_recurrent, layer  10: dev = MTL0\nllama_memory_recurrent, layer  11: dev = MTL0\nllama_memory_recurrent, layer  12: dev = MTL0\nllama_memory_recurrent, layer  13: dev = MTL0\nllama_memory_recurrent, layer  14: dev = MTL0\nllama_memory_recurrent, layer  16: dev = MTL0\nllama_memory_recurrent, layer  17: dev = MTL0\nllama_memory_recurrent, layer  18: dev = MTL0\nllama_memory_recurrent, layer  19: dev = MTL0\nllama_memory_recurrent, layer  20: dev = MTL0\nllama_memory_recurrent, layer  21: dev = MTL0\nllama_memory_recurrent, layer  22: dev = MTL0\nllama_memory_recurrent, layer  23: dev = MTL0\nllama_memory_recurrent, layer  24: dev = MTL0\nllama_memory_recurrent, layer  26: dev = MTL0\nllama_memory_recurrent, layer  27: dev = MTL0\nllama_memory_recurrent, layer  28: dev = MTL0\nllama_memory_recurrent, layer  29: dev = MTL0\nllama_memory_recurrent, layer  30: dev = MTL0\nllama_memory_recurrent, layer  31: dev = MTL0\nllama_memory_recurrent, layer  32: dev = MTL0\nllama_memory_recurrent, layer  33: dev = MTL0\nllama_memory_recurrent, layer  34: dev = MTL0\nllama_memory_recurrent, layer  36: dev = MTL0\nllama_memory_recurrent, layer  37: dev = MTL0\nllama_memory_recurrent, layer  38: dev = MTL0\nllama_memory_recurrent, layer  39: dev = MTL0\nllama_memory_recurrent:       MTL0 RS buffer size =    55.37 MiB\nsched_reserve:       MTL0 compute buffer size =   123.63 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 41,
      "totalLayers": 41,
      "inferenceMs": 124.71774999721674,
      "durationMs": 124.92416700115427,
      "tokensPerSecond": 64.14483904799863,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 4531126272,
      "cpuThreads": 4,
      "totalMs": 511.3007510008174,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 4299.669874999381,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =  4029.02 MiB\nload_tensors:   CPU_REPACK model buffer size =  4013.35 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 210.0615840026876,
      "durationMs": 210.47516699763946,
      "tokensPerSecond": 38.084069669291104,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 8732213248,
      "cpuThreads": 4,
      "totalMs": 4510.145041997021,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 4299.669874999381,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =  4029.02 MiB\nload_tensors:   CPU_REPACK model buffer size =  4013.35 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 206.4705419979873,
      "durationMs": 206.67079099803232,
      "tokensPerSecond": 38.74644742337134,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 8734425088,
      "cpuThreads": 4,
      "totalMs": 4506.340665997413,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 4299.669874999381,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =  4029.02 MiB\nload_tensors:   CPU_REPACK model buffer size =  4013.35 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 222.43145799438935,
      "durationMs": 222.71387500222772,
      "tokensPerSecond": 35.96613569022145,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 8737112064,
      "cpuThreads": 4,
      "totalMs": 4522.383750001609,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 6856.6297910001595,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =  4029.02 MiB\nload_tensors:   CPU_REPACK model buffer size =  4013.35 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 1210.2847919959459,
      "durationMs": 1211.5372080006637,
      "tokensPerSecond": 6.610014479985962,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 4040474624,
      "cpuThreads": 4,
      "totalMs": 8068.166999000823,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 6856.6297910001595,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =  4029.02 MiB\nload_tensors:   CPU_REPACK model buffer size =  4013.35 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 810.4994159948546,
      "durationMs": 811.4228330014157,
      "tokensPerSecond": 9.870457451447178,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 4562386944,
      "cpuThreads": 4,
      "totalMs": 7668.052624001575,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 6856.6297910001595,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =  4029.02 MiB\nload_tensors:   CPU_REPACK model buffer size =  4013.35 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 778.7719580010162,
      "durationMs": 780.087041996012,
      "tokensPerSecond": 10.27258354362775,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 5371920384,
      "cpuThreads": 4,
      "totalMs": 7636.716832996171,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 6245.359791995725,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =  4029.02 MiB\nload_tensors:   CPU_REPACK model buffer size =  4013.35 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 568.5241249957471,
      "durationMs": 568.9494170001126,
      "tokensPerSecond": 14.071522470677643,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 6858375168,
      "cpuThreads": 4,
      "totalMs": 6814.309208995837,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 6245.359791995725,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =  4029.02 MiB\nload_tensors:   CPU_REPACK model buffer size =  4013.35 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 658.1361670032493,
      "durationMs": 658.3746670003165,
      "tokensPerSecond": 12.155539235029615,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 7467794432,
      "cpuThreads": 4,
      "totalMs": 6903.734458996041,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 6245.359791995725,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =  4029.02 MiB\nload_tensors:   CPU_REPACK model buffer size =  4013.35 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 2131.517124995298,
      "durationMs": 2131.909582996741,
      "tokensPerSecond": 3.7531952740082475,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 7499988992,
      "cpuThreads": 4,
      "totalMs": 8377.269374992466,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": false,
      "expected": " France"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 120087.31554199767,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =  4029.02 MiB\nload_tensors:   CPU_REPACK model buffer size =  4013.35 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 12897.468124996522,
      "durationMs": 12919.396457997209,
      "tokensPerSecond": 0.6202767800988194,
      "inputTokens": 8,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 2466267136,
      "cpuThreads": 1,
      "totalMs": 133006.71199999488,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": false,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 120087.31554199767,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =  4029.02 MiB\nload_tensors:   CPU_REPACK model buffer size =  4013.35 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 4404.857874993468,
      "durationMs": 4412.677291002183,
      "tokensPerSecond": 1.8161766456566693,
      "inputTokens": 11,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 3198730240,
      "cpuThreads": 1,
      "totalMs": 124499.99283299985,
      "grade": "slow",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "llama.cpp",
      "version": "0.3.36",
      "artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
      "device": "cpu",
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "loadMs": 120087.31554199767,
      "acceleration": "cpu",
      "placement": "ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors:   CPU_Mapped model buffer size =  4029.02 MiB\nload_tensors:   CPU_REPACK model buffer size =  4013.35 MiB",
      "parameterDtype": "Q4_K_M mixed tensor types",
      "offloadedLayers": 0,
      "totalLayers": 41,
      "inferenceMs": 3804.8443329971633,
      "durationMs": 3814.379124996776,
      "tokensPerSecond": 2.1025827339690966,
      "inputTokens": 16,
      "outputTokens": 8,
      "outputLimitReached": true,
      "rssBytes": 3765305344,
      "cpuThreads": 1,
      "totalMs": 123901.69466699444,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  }
]

Granite 4.0 Dense Micro Base

21 configurations tested
Passed21
Failed0
TestedNov 7, 2026
Runtimetransformers 5.22.1

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedFastmacos · aarch64298 ms / 1870 ms
FP16mpsAcceleratedPassedFastmacos · aarch64299 ms / 1873 ms
FP16mpsAcceleratedPassedFastmacos · aarch64306 ms / 1873 ms
FP16mpsAcceleratedPassedFastmacos · aarch64306 ms / 1870 ms
FP16mpsAcceleratedPassedFastmacos · aarch64310 ms / 1870 ms
FP16mpsAcceleratedPassedFastmacos · aarch64314 ms / 3088 ms
FP16mpsAcceleratedPassedFastmacos · aarch64316 ms / 1873 ms
FP16mpsAcceleratedPassedFastmacos · aarch64390 ms / 3088 ms
FP16mpsAcceleratedPassedModeratemacos · aarch64450 ms / 3088 ms
FP32cpuPassedModeratemacos · aarch641390 ms / 1822 ms
FP32cpuPassedModeratemacos · aarch641451 ms / 2097 ms
FP32cpuPassedModeratemacos · aarch641475 ms / 2097 ms
FP32cpuPassedModeratemacos · aarch641501 ms / 1716 ms
FP32cpuPassedModeratemacos · aarch641559 ms / 1939 ms
FP32cpuPassedModeratemacos · aarch641577 ms / 1822 ms
FP32cpuPassedModeratemacos · aarch641601 ms / 2097 ms
FP32cpuPassedModeratemacos · aarch641609 ms / 1939 ms
FP32cpuPassedModeratemacos · aarch641622 ms / 1716 ms
FP32cpuPassedModeratemacos · aarch641663 ms / 1939 ms
FP32cpuPassedModeratemacos · aarch641724 ms / 1822 ms
FP32cpuPassedModeratemacos · aarch641779 ms / 1716 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-07T12:42:20.217541+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 5.22.1
Artifact revision
98e1d641451818f6c8e6b17c1c359b5b4ac495d0
Fixture
{
  "file": "model-samples/desktop/granite4/granite-4.0-micro-base-native-geography.json",
  "size": 152,
  "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef",
  "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21",
  "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-micro-base-native-geography.json"
}
Rerun when
Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 3087.7746249971096,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 322
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 451.4157500016154,
      "inferenceMs": 450.1381250011036,
      "tokensPerSecond": 17.77232266202821,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7544389632,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2111815680,
      "cpuThreads": 4,
      "totalMs": 3539.190374998725,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 3087.7746249971096,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 322
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 392.0737910011667,
      "inferenceMs": 390.0102909974521,
      "tokensPerSecond": 20.512279251760212,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7544487936,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2121678848,
      "cpuThreads": 4,
      "totalMs": 3479.8484159982763,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 3087.7746249971096,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 322
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 315.25812500331085,
      "inferenceMs": 313.62633299431764,
      "tokensPerSecond": 25.508062169463766,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7553024000,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2131476480,
      "cpuThreads": 4,
      "totalMs": 3403.0327500004205,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1869.5750420010881,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 322
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 310.65529100305866,
      "inferenceMs": 309.6002500024042,
      "tokensPerSecond": 25.839772416003786,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7544389632,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2105032704,
      "cpuThreads": 4,
      "totalMs": 2180.230333004147,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1869.5750420010881,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 322
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 299.4262920037727,
      "inferenceMs": 297.85954100225354,
      "tokensPerSecond": 26.85829694453022,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7544487936,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2116059136,
      "cpuThreads": 4,
      "totalMs": 2169.001334004861,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1869.5750420010881,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 322
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 307.621833999292,
      "inferenceMs": 305.8280829936848,
      "tokensPerSecond": 26.158487218340888,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7553024000,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2126413824,
      "cpuThreads": 4,
      "totalMs": 2177.19687600038,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1873.33916699572,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 322
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 308.6056250031106,
      "inferenceMs": 305.5762500007404,
      "tokensPerSecond": 26.180045078701685,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7544389632,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2107883520,
      "cpuThreads": 4,
      "totalMs": 2181.9447919988306,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1873.33916699572,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 322
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 300.56641699775355,
      "inferenceMs": 299.0010830035317,
      "tokensPerSecond": 26.7557559311767,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7544487936,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2118598656,
      "cpuThreads": 4,
      "totalMs": 2173.9055839934736,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1873.33916699572,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 322
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 318.0984159989748,
      "inferenceMs": 316.37737499841023,
      "tokensPerSecond": 25.286258222605834,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "driverAllocatedMemoryBytes": 7553024000,
      "memoryMeasurementScope": "Post-inference driver allocation; not peak",
      "rssBytes": 2128871424,
      "cpuThreads": 4,
      "totalMs": 2191.437582994695,
      "grade": "fast",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 2096.5990420008893,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1451.0336670064135,
      "inferenceMs": 1450.7435410050675,
      "tokensPerSecond": 5.514413660223944,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 14042464256,
      "cpuThreads": 4,
      "totalMs": 3547.6327090073028,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 2096.5990420008893,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1475.6915000034496,
      "inferenceMs": 1475.4095419993973,
      "tokensPerSecond": 5.422223302933788,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 14044594176,
      "cpuThreads": 4,
      "totalMs": 3572.290542004339,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 2096.5990420008893,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1601.0853330008104,
      "inferenceMs": 1600.7957500041812,
      "tokensPerSecond": 4.997514517376189,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 14047526912,
      "cpuThreads": 4,
      "totalMs": 3697.6843750016997,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1939.3144589994336,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1558.9824580019922,
      "inferenceMs": 1558.7354169983882,
      "tokensPerSecond": 5.132365578377226,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 14041481216,
      "cpuThreads": 4,
      "totalMs": 3498.2969170014258,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1939.3144589994336,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1663.5973749944242,
      "inferenceMs": 1663.3482499964884,
      "tokensPerSecond": 4.809576106517014,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 14044037120,
      "cpuThreads": 4,
      "totalMs": 3602.9118339938577,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1939.3144589994336,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1609.4265830033692,
      "inferenceMs": 1609.0771660019527,
      "tokensPerSecond": 4.9717938760373235,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 14048477184,
      "cpuThreads": 4,
      "totalMs": 3548.7410420028027,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1716.4740000007441,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1779.475708004611,
      "inferenceMs": 1779.2273339946405,
      "tokensPerSecond": 4.496333800155129,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 14041350144,
      "cpuThreads": 4,
      "totalMs": 3495.949708005355,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1716.4740000007441,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1621.980750001967,
      "inferenceMs": 1621.699333001743,
      "tokensPerSecond": 4.933096929374763,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 14042857472,
      "cpuThreads": 4,
      "totalMs": 3338.454750002711,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1716.4740000007441,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1501.2385829977575,
      "inferenceMs": 1500.9189170013997,
      "tokensPerSecond": 5.330068073219268,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 14046232576,
      "cpuThreads": 4,
      "totalMs": 3217.7125829985016,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "plainParity": true,
      "passed": true,
      "expected": " France"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1821.595375004108,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1576.9694170012372,
      "inferenceMs": 1576.7008750044624,
      "tokensPerSecond": 5.073885685499704,
      "inputTokens": 8,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 14037549056,
      "cpuThreads": 1,
      "totalMs": 3398.564792005345,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " Celsius"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1821.595375004108,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1723.7836250060354,
      "inferenceMs": 1723.5318749953876,
      "tokensPerSecond": 4.641631591537237,
      "inputTokens": 11,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 14040924160,
      "cpuThreads": 1,
      "totalMs": 3545.3790000101435,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedText": true,
      "streamParity": true,
      "withinTokenBudget": true,
      "passed": true,
      "expected": " x"
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
      "torchVersion": "2.10.0",
      "loadMs": 1821.595375004108,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 322
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
      "durationMs": 1390.4377500002738,
      "inferenceMs": 1390.0973749987315,
      "tokensPerSecond": 5.754992523460667,
      "inputTokens": 16,
      "outputTokens": 8,
      "maxNewTokens": 8,
      "decoding": "greedy",
      "outputLimitReached": true,
      "rssBytes": 14045429760,
      "cpuThreads": 1,
      "totalMs": 3212.033125004382,
      "grade": "acceptable",
      "loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
    }
  }
]
Release validation still required
  • Verified public B2 backup and separate model/runtime source-outage recovery pending.

Something not quite right?

Tell us what happened with Granite 4.0. Loading, results, or the page itself—we’re here for all of it.

Attachments Optional

Up to 3 files · 50 MB each

Browser, OS, CPU/GPU, memory, screen, storage, and network details are included automatically. Model inputs, results, and browsing history are not collected.