EmbeddingGemma 300M
12 configurations testedQuality criteria
Finite normalized vectors with the correct dimensions, distinct rows, and the relevant-document cosine exceeding the unrelated-document cosine by at least 0.1.
| Precision | Device | Result | |
|---|---|---|---|
Q4 | GPU · WebGPUAccelerated | Passed | Very fast |
Q8 | GPU · WebGPUAccelerated | Passed | Fast |
NO_GATHER_Q4 | GPU · WebGPUAccelerated | Passed | Fast |
FP32 | GPU · WebGPUAccelerated | Passed | Fast |
FP32 | CPU · WASM | Passed | Fast |
FP16 | CPU · WASM | Passed | Fast |
Q8 | CPU · WASM | Passed | Fast |
NO_GATHER_Q4 | CPU · WASM | Passed | Moderate |
FP16 | GPU · WebGPU | Failed | — |
Q4 | CPU · WASM | Failed | — |
Q4F16 | CPU · WASM | Failed | — |
Q4F16 | GPU · WebGPU | Failed | — |
Test setup & sample details
- Quality gate
- Finite normalized vectors with the correct dimensions, distinct rows, and the relevant-document cosine exceeding the unrelated-document cosine by at least 0.1.
- Tested
- 2026-09-12T14:42:07.406Z
- Environment
- Not recorded
- Runtime
- Transformers.js 4.2.0
- Artifact revision
- 5090578d9565bb06545b4552f76e6bc2c93e4a66
- Fixture
{ "file": "embeddinggemma-300m.json", "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/embeddinggemma-300m.json", "size": 634, "sha1": "9a1d261338de718b559f5544e81e488477672779" }- Rerun when
- artifact revision change; runtime version change; fixture or quality-gate change; browser/backend behavior change
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "wasm",
"status": "pass",
"quality": {
"margin": 0.6141
},
"runtime": {
"loadMs": 140152.69999999553,
"inferenceMs": 320.6000000014901,
"totalMs": 141775.79999999702,
"embeddingsPerSecond": 9.3575,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "webgpu",
"status": "pass",
"quality": {
"margin": 0.6141
},
"runtime": {
"loadMs": 159880.8999999985,
"inferenceMs": 251.30000000447035,
"totalMs": 161532.29999999702,
"embeddingsPerSecond": 11.9379,
"grade": "good"
}
},
{
"precision": "fp16",
"device": "wasm",
"status": "pass",
"quality": {
"margin": 0.6141
},
"runtime": {
"loadMs": 85957.60000000149,
"inferenceMs": 337.70000000298023,
"totalMs": 87670.39999999851,
"embeddingsPerSecond": 8.8836,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "webgpu",
"status": "fail",
"quality": "Not embedded",
"runtime": "Not embedded",
"failure": "WebGPU LayerNorm shader validation failed during inference."
},
{
"precision": "q8",
"device": "wasm",
"status": "pass",
"quality": {
"margin": 0.6085
},
"runtime": {
"loadMs": 49854.39999999851,
"inferenceMs": 445.6000000014901,
"totalMs": 51707.5,
"embeddingsPerSecond": 6.7325,
"grade": "slow"
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "pass",
"quality": {
"margin": 0.6085
},
"runtime": {
"loadMs": 46093.30000000447,
"inferenceMs": 174.89999999850988,
"totalMs": 47672.39999999851,
"embeddingsPerSecond": 17.1527,
"grade": "good"
}
},
{
"precision": "q4",
"device": "wasm",
"status": "fail",
"quality": "Not embedded",
"runtime": "Not embedded",
"failure": "WASM has no implementation for GatherBlockQuantized."
},
{
"precision": "q4",
"device": "webgpu",
"status": "pass",
"quality": {
"margin": 0.5983
},
"runtime": {
"loadMs": 36447.19999999553,
"inferenceMs": 80.60000000149012,
"totalMs": 37903.39999999851,
"embeddingsPerSecond": 37.2208,
"grade": "excellent"
}
},
{
"precision": "q4f16",
"device": "wasm",
"status": "fail",
"quality": "Not embedded",
"runtime": "Not embedded",
"failure": "WASM has no GatherBlockQuantized kernel."
},
{
"precision": "q4f16",
"device": "webgpu",
"status": "fail",
"quality": "Not embedded",
"runtime": "Not embedded",
"failure": "WebGPU LayerNorm shader validation failed during inference."
},
{
"precision": "no_gather_q4",
"device": "wasm",
"status": "pass",
"quality": {
"margin": 0.5969
},
"runtime": {
"loadMs": 28191.29999999702,
"inferenceMs": 1558.5,
"totalMs": 30994,
"embeddingsPerSecond": 1.9249,
"grade": "very-slow"
}
},
{
"precision": "no_gather_q4",
"device": "webgpu",
"status": "pass",
"quality": {
"margin": 0.5969
},
"runtime": {
"loadMs": 26292,
"inferenceMs": 188,
"totalMs": 27748.89999999851,
"embeddingsPerSecond": 15.9574,
"grade": "good"
}
}
]