BGE-M3 — Multilingual Embeddings
16 configurations testedQuality criteria
Finite normalized 1024d vectors, distinct rows, and positive cosine exceeds negative by at least 0.1.
| Precision | Device | Result | |
|---|---|---|---|
FP16 | GPU · WebGPUAccelerated | Passed | Fast |
Q4F16 | GPU · WebGPUAccelerated | Passed | Moderate |
Q8 | GPU · WebGPUAccelerated | Passed | Moderate |
UINT8 | GPU · WebGPUAccelerated | Passed | Moderate |
INT8 | GPU · WebGPUAccelerated | Passed | Moderate |
INT8 | CPU · WASM | Passed | Moderate |
Q8 | CPU · WASM | Passed | Moderate |
UINT8 | CPU · WASM | Passed | Moderate |
BNB4 | CPU · WASM | Passed | Moderate |
Q4 | CPU · WASM | Passed | Moderate |
Q4F16 | CPU · WASM | Failed | — |
Q4 | GPU · WebGPU | Failed | — |
BNB4 | GPU · WebGPU | Failed | — |
FP16 | CPU · WASM | Failed | — |
FP32 | CPU · WASM | Failed | — |
FP32 | GPU · WebGPU | Failed | — |
Test setup & sample details
- Quality gate
- Finite normalized 1024d vectors, distinct rows, and positive cosine exceeds negative by at least 0.1.
- Tested
- 2026-09-10T10:58:00.000Z
- Environment
- Not recorded
- Runtime
- Transformers.js 4.2.0
- Artifact revision
- 4de13258303883538bd53b696b452bf8099f0858
- Fixture
{ "file": "bge-m3.json", "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/bge-m3.json", "size": 500, "sha1": "75a7770d06fc312ddeb39987712a8b4bbe6f1e08" }- Rerun when
- artifact revision change; runtime version change; fixture or quality gate change; browser/backend change
Per-case quality & failure details
[
{
"precision": "q8",
"device": "wasm",
"status": "pass",
"quality": "Not embedded",
"runtime": {
"loadMs": 66888.29999995232,
"inferenceMs": 668.8000000715256,
"totalMs": 68783.10000002384,
"embeddingsPerSecond": 4.485645932534632,
"grade": "slow"
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "pass",
"quality": "Not embedded",
"runtime": {
"loadMs": 68687.79999995232,
"inferenceMs": 774.3999999761581,
"totalMs": 70737.19999992847,
"embeddingsPerSecond": 3.87396694226803,
"grade": "slow"
}
},
{
"precision": "int8",
"device": "wasm",
"status": "pass",
"quality": "Not embedded",
"runtime": {
"loadMs": 68673.5,
"inferenceMs": 667.2000000476837,
"totalMs": 70675.60000002384,
"embeddingsPerSecond": 4.49640287737649,
"grade": "slow"
}
},
{
"precision": "int8",
"device": "webgpu",
"status": "pass",
"quality": "Not embedded",
"runtime": {
"loadMs": 67158.39999997616,
"inferenceMs": 841.2000000476837,
"totalMs": 69326.39999997616,
"embeddingsPerSecond": 3.5663338086423493,
"grade": "slow"
}
},
{
"precision": "uint8",
"device": "wasm",
"status": "pass",
"quality": "Not embedded",
"runtime": {
"loadMs": 70235.89999997616,
"inferenceMs": 679.4000000953674,
"totalMs": 72221.09999990463,
"embeddingsPerSecond": 4.415660876624801,
"grade": "slow"
}
},
{
"precision": "uint8",
"device": "webgpu",
"status": "pass",
"quality": "Not embedded",
"runtime": {
"loadMs": 90571.5,
"inferenceMs": 775.6000000238419,
"totalMs": 92608.80000007153,
"embeddingsPerSecond": 3.8679731819337033,
"grade": "slow"
}
},
{
"precision": "q4f16",
"device": "wasm",
"status": "fail",
"quality": "Not embedded",
"runtime": "Not embedded",
"failure": "ONNX session graph error"
},
{
"precision": "q4f16",
"device": "webgpu",
"status": "pass",
"quality": "Not embedded",
"runtime": {
"loadMs": 121106.29999995232,
"inferenceMs": 513,
"totalMs": 122968,
"embeddingsPerSecond": 5.847953216374269,
"grade": "slow"
}
},
{
"precision": "q4",
"device": "wasm",
"status": "pass",
"quality": "Not embedded",
"runtime": {
"loadMs": 249183.79999995232,
"inferenceMs": 996.6000000238419,
"totalMs": 251455.89999997616,
"embeddingsPerSecond": 3.010234798242254,
"grade": "slow"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "fail",
"quality": "Not embedded",
"runtime": "Not embedded",
"failure": "WebGPU table index out of bounds"
},
{
"precision": "bnb4",
"device": "wasm",
"status": "pass",
"quality": "Not embedded",
"runtime": {
"loadMs": 133445.09999990463,
"inferenceMs": 801.4000000953674,
"totalMs": 135454,
"embeddingsPerSecond": 3.743448963866979,
"grade": "slow"
}
},
{
"precision": "bnb4",
"device": "webgpu",
"status": "fail",
"quality": "Not embedded",
"runtime": "Not embedded",
"failure": "WebGPU table index out of bounds"
},
{
"precision": "fp16",
"device": "wasm",
"status": "fail",
"quality": "Not embedded",
"runtime": "Not embedded",
"failure": "ONNX session graph error"
},
{
"precision": "fp16",
"device": "webgpu",
"status": "pass",
"quality": "Not embedded",
"runtime": {
"loadMs": 123899.80000007153,
"inferenceMs": 247.79999995231628,
"totalMs": 125337.10000002384,
"embeddingsPerSecond": 12.106537532595985,
"grade": "good"
}
},
{
"precision": "fp32",
"device": "wasm",
"status": "fail",
"quality": "Not embedded",
"runtime": "Not embedded",
"failure": "bounded browser timeout"
},
{
"precision": "fp32",
"device": "webgpu",
"status": "fail",
"quality": "Not embedded",
"runtime": "Not embedded",
"failure": "bounded browser timeout"
}
]