MVANet
7 configurations testedQuality criteria
{ "frozenAt": "2026-11-16T13:20:00Z", "scope": "Existing independently authored foreground/background point annotations on two natural photographs. Portrait alpha IoU/MAE/boundaryF1 separately measures MODNet reference agreement, not human alpha truth.", "source": "research/slimsam-family-expansion/quality-gates.json", "sourceSha256": "2a1a8eddb1ab2cb5d82d99cc722f0d19d9375a90536b8a0a56fb9c36826abd4d", "minimumProbeAccuracy": 0.9, "maximumInferenceMs": 10000, "cases": [ { "file": "model-samples/modnet-portrait-pexels-5965592.jpg", "expected": { "width": 1024, "height": 683, "point": [ 470, 330 ], "minimumForegroundRatio": 0.15, "maximumForegroundRatio": 0.4, "foregroundPoints": [ [ 450, 120 ], [ 460, 210 ], [ 430, 350 ], [ 350, 420 ], [ 460, 520 ], [ 480, 630 ], [ 580, 470 ] ], "backgroundPoints": [ [ 50, 350 ], [ 800, 350 ], [ 900, 100 ], [ 100, 600 ], [ 700, 600 ], [ 200, 200 ] ], "minimumProbeAccuracy": 0.9 }, "source": "WebAI independently annotated whole-person foreground/background probes on the Pexels portrait, visually reviewed at original 1024x683 resolution.", "license": "Pexels License", "id": "person-box", "userInput": { "input_boxes": [ [ 230, 40, 740, 683 ] ] } }, { "file": "model-samples/desktop/mobilenet-dog.jpg", "expected": { "minimumForegroundRatio": 0.25, "maximumForegroundRatio": 0.8, "foregroundPoints": [ [ 200, 150 ], [ 350, 200 ], [ 500, 400 ] ], "backgroundPoints": [ [ 0, 0 ], [ 690, 570 ], [ 680, 10 ] ], "minimumProbeAccuracy": 0.9 }, "source": "https://commons.wikimedia.org/wiki/File:YellowLabradorLooking_new.jpg — Elf; derivative Djmirko/FT2; independent visual foreground/background probes.", "license": "CC-BY-SA-3.0", "id": "dog-box", "userInput": { "input_boxes": [ [ 50, 30, 655, 560 ] ] } } ], "familyId": "mvanet", "runtime": "Transformers.js4.3.2 preferred browser; pinned native Transformers architecture evaluated before ONNX/PyTorch alternatives", "preprocess": "Pinned processor bilinear1024/ImageNet normalization; exact graph signatures determine sigmoid handling. No BEN2 timing reuse." }
| Precision | Device | Result | |
|---|---|---|---|
Q4F16 | GPU · WebGPUAccelerated | Passed | Moderate |
Q4F16 | GPU · WebGPUAccelerated | Passed | Moderate |
Q4F16 | GPU · WebGPUAccelerated | Passed | Moderate |
Q4F16 | GPU · WebGPUAccelerated | Passed | Moderate |
Q4F16 | GPU · WebGPUAccelerated | Passed | Moderate |
Q4F16 | GPU · WebGPUAccelerated | Passed | Moderate |
Q4F16 | GPU · WebGPUAccelerated | Passed | Moderate |
Test setup & sample details
- Quality gate
- { "frozenAt": "2026-11-16T13:20:00Z", "scope": "Existing independently authored foreground/background point annotations on two natural photographs. Portrait alpha IoU/MAE/boundaryF1 separately measures MODNet reference agreement, not human alpha truth.", "source": "research/slimsam-family-expansion/quality-gates.json", "sourceSha256": "2a1a8eddb1ab2cb5d82d99cc722f0d19d9375a90536b8a0a56fb9c36826abd4d", "minimumProbeAccuracy": 0.9, "maximumInferenceMs": 10000, "cases": [ { "file": "model-samples/modnet-portrait-pexels-5965592.jpg", "expected": { "width": 1024, "height": 683, "point": [ 470, 330 ], "minimumForegroundRatio": 0.15, "maximumForegroundRatio": 0.4, "foregroundPoints": [ [ 450, 120 ], [ 460, 210 ], [ 430, 350 ], [ 350, 420 ], [ 460, 520 ], [ 480, 630 ], [ 580, 470 ] ], "backgroundPoints": [ [ 50, 350 ], [ 800, 350 ], [ 900, 100 ], [ 100, 600 ], [ 700, 600 ], [ 200, 200 ] ], "minimumProbeAccuracy": 0.9 }, "source": "WebAI independently annotated whole-person foreground/background probes on the Pexels portrait, visually reviewed at original 1024x683 resolution.", "license": "Pexels License", "id": "person-box", "userInput": { "input_boxes": [ [ 230, 40, 740, 683 ] ] } }, { "file": "model-samples/desktop/mobilenet-dog.jpg", "expected": { "minimumForegroundRatio": 0.25, "maximumForegroundRatio": 0.8, "foregroundPoints": [ [ 200, 150 ], [ 350, 200 ], [ 500, 400 ] ], "backgroundPoints": [ [ 0, 0 ], [ 690, 570 ], [ 680, 10 ] ], "minimumProbeAccuracy": 0.9 }, "source": "https://commons.wikimedia.org/wiki/File:YellowLabradorLooking_new.jpg — Elf; derivative Djmirko/FT2; independent visual foreground/background probes.", "license": "CC-BY-SA-3.0", "id": "dog-box", "userInput": { "input_boxes": [ [ 50, 30, 655, 560 ] ] } } ], "familyId": "mvanet", "runtime": "Transformers.js4.3.2 preferred browser; pinned native Transformers architecture evaluated before ONNX/PyTorch alternatives", "preprocess": "Pinned processor bilinear1024/ImageNet normalization; exact graph signatures determine sigmoid handling. No BEN2 timing reuse." }
- Tested
- 2026-11-16T13:21:34.280Z
- Environment
- Apple M3 Max macOS27.2 Chrome157; original staged exact artifacts; existing background mirror active
- Runtime
- transformers.js 4.3.2
- Artifact revision
- 43ec3427514b8d9164eea02df93ca2f1b036bb7b
- Fixture
{ "file": "model-samples/modnet-portrait-pexels-5965592.jpg", "size": 78124, "sha1": "38212739a4a4f0be25d84863cf991d7fb6c6d818", "sha256": "2b5d6423b8948eb53319661633039912afa6455a9055f0c984ff88eeae1b5bfe", "url": "https://assets.localai.tools/samples/v1/files/2b5d6423b8948eb53319661633039912afa6455a9055f0c984ff88eeae1b5bfe/modnet-portrait-pexels-5965592.jpg" }- Rerun when
- artifact revision; runtime version; fixture checksum; quality gate; backend behavior
Per-case quality & failure details
[
{
"precision": "q4f16",
"device": "webgpu",
"status": "pass",
"quality": {
"platform": "browser",
"id": "portrait",
"source": "research/mvanet/browser-public-webgpu.json",
"runtime": {
"durationMs": 764.0050000026822,
"inferenceMs": 764.1200000010431,
"loadMs": 3490.719999998808
},
"probeAccuracy": 1,
"foregroundRatio": 0.2587576066068814,
"probes": [
{
"point": [
450,
120
],
"foreground": true,
"passed": true
},
{
"point": [
460,
210
],
"foreground": true,
"passed": true
},
{
"point": [
430,
350
],
"foreground": true,
"passed": true
},
{
"point": [
350,
420
],
"foreground": true,
"passed": true
},
{
"point": [
460,
520
],
"foreground": true,
"passed": true
},
{
"point": [
480,
630
],
"foreground": true,
"passed": true
},
{
"point": [
580,
470
],
"foreground": true,
"passed": true
},
{
"point": [
50,
350
],
"foreground": false,
"passed": true
},
{
"point": [
800,
350
],
"foreground": false,
"passed": true
},
{
"point": [
900,
100
],
"foreground": false,
"passed": true
},
{
"point": [
100,
600
],
"foreground": false,
"passed": true
},
{
"point": [
700,
600
],
"foreground": false,
"passed": true
},
{
"point": [
200,
200
],
"foreground": false,
"passed": true
}
],
"passed": true,
"modnetReferenceAgreement": {
"iou": 0.9850542728105918,
"alphaMAE": 0.005443826391281256,
"note": "Model-reference agreement, not human ground truth"
}
},
"runtime": {
"durationMs": 764.0050000026822,
"inferenceMs": 764.1200000010431,
"loadMs": 3490.719999998808,
"totalMs": 4254.839999999851,
"imagesPerSecond": 1.3086949693747512,
"grade": "good",
"engine": "transformers.js",
"version": "4.3.2",
"actualDevice": "webgpu",
"actualPrecision": "q4f16",
"artifactRevision": "43ec3427514b8d9164eea02df93ca2f1b036bb7b"
}
},
{
"precision": "q4f16",
"device": "webgpu",
"status": "pass",
"quality": {
"platform": "browser",
"id": "portrait-repeat-1",
"source": "research/mvanet/browser-public-webgpu.json",
"runtime": {
"durationMs": 721.5300000011921,
"inferenceMs": 721.5600000023842,
"loadMs": 3490.719999998808
},
"probeAccuracy": 1,
"foregroundRatio": 0.2587576066068814,
"probes": [
{
"point": [
450,
120
],
"foreground": true,
"passed": true
},
{
"point": [
460,
210
],
"foreground": true,
"passed": true
},
{
"point": [
430,
350
],
"foreground": true,
"passed": true
},
{
"point": [
350,
420
],
"foreground": true,
"passed": true
},
{
"point": [
460,
520
],
"foreground": true,
"passed": true
},
{
"point": [
480,
630
],
"foreground": true,
"passed": true
},
{
"point": [
580,
470
],
"foreground": true,
"passed": true
},
{
"point": [
50,
350
],
"foreground": false,
"passed": true
},
{
"point": [
800,
350
],
"foreground": false,
"passed": true
},
{
"point": [
900,
100
],
"foreground": false,
"passed": true
},
{
"point": [
100,
600
],
"foreground": false,
"passed": true
},
{
"point": [
700,
600
],
"foreground": false,
"passed": true
},
{
"point": [
200,
200
],
"foreground": false,
"passed": true
}
],
"passed": true,
"modnetReferenceAgreement": {
"iou": 0.9850542728105918,
"alphaMAE": 0.005443826391281256,
"note": "Model-reference agreement, not human ground truth"
}
},
"runtime": {
"durationMs": 721.5300000011921,
"inferenceMs": 721.5600000023842,
"loadMs": 3490.719999998808,
"totalMs": 4212.280000001192,
"imagesPerSecond": 1.3858861355905203,
"grade": "good",
"engine": "transformers.js",
"version": "4.3.2",
"actualDevice": "webgpu",
"actualPrecision": "q4f16",
"artifactRevision": "43ec3427514b8d9164eea02df93ca2f1b036bb7b"
}
},
{
"precision": "q4f16",
"device": "webgpu",
"status": "pass",
"quality": {
"platform": "browser",
"id": "portrait-repeat-2",
"source": "research/mvanet/browser-public-webgpu.json",
"runtime": {
"durationMs": 704.7899999991059,
"inferenceMs": 704.804999999702,
"loadMs": 3490.719999998808
},
"probeAccuracy": 1,
"foregroundRatio": 0.2587576066068814,
"probes": [
{
"point": [
450,
120
],
"foreground": true,
"passed": true
},
{
"point": [
460,
210
],
"foreground": true,
"passed": true
},
{
"point": [
430,
350
],
"foreground": true,
"passed": true
},
{
"point": [
350,
420
],
"foreground": true,
"passed": true
},
{
"point": [
460,
520
],
"foreground": true,
"passed": true
},
{
"point": [
480,
630
],
"foreground": true,
"passed": true
},
{
"point": [
580,
470
],
"foreground": true,
"passed": true
},
{
"point": [
50,
350
],
"foreground": false,
"passed": true
},
{
"point": [
800,
350
],
"foreground": false,
"passed": true
},
{
"point": [
900,
100
],
"foreground": false,
"passed": true
},
{
"point": [
100,
600
],
"foreground": false,
"passed": true
},
{
"point": [
700,
600
],
"foreground": false,
"passed": true
},
{
"point": [
200,
200
],
"foreground": false,
"passed": true
}
],
"passed": true,
"modnetReferenceAgreement": {
"iou": 0.9850542728105918,
"alphaMAE": 0.005443826391281256,
"note": "Model-reference agreement, not human ground truth"
}
},
"runtime": {
"durationMs": 704.7899999991059,
"inferenceMs": 704.804999999702,
"loadMs": 3490.719999998808,
"totalMs": 4195.52499999851,
"imagesPerSecond": 1.4188321592503215,
"grade": "good",
"engine": "transformers.js",
"version": "4.3.2",
"actualDevice": "webgpu",
"actualPrecision": "q4f16",
"artifactRevision": "43ec3427514b8d9164eea02df93ca2f1b036bb7b"
}
},
{
"precision": "q4f16",
"device": "webgpu",
"status": "pass",
"quality": {
"platform": "browser",
"id": "portrait-repeat-3",
"source": "research/mvanet/browser-public-webgpu.json",
"runtime": {
"durationMs": 705.2900000028312,
"inferenceMs": 705.3149999976158,
"loadMs": 3490.719999998808
},
"probeAccuracy": 1,
"foregroundRatio": 0.2587576066068814,
"probes": [
{
"point": [
450,
120
],
"foreground": true,
"passed": true
},
{
"point": [
460,
210
],
"foreground": true,
"passed": true
},
{
"point": [
430,
350
],
"foreground": true,
"passed": true
},
{
"point": [
350,
420
],
"foreground": true,
"passed": true
},
{
"point": [
460,
520
],
"foreground": true,
"passed": true
},
{
"point": [
480,
630
],
"foreground": true,
"passed": true
},
{
"point": [
580,
470
],
"foreground": true,
"passed": true
},
{
"point": [
50,
350
],
"foreground": false,
"passed": true
},
{
"point": [
800,
350
],
"foreground": false,
"passed": true
},
{
"point": [
900,
100
],
"foreground": false,
"passed": true
},
{
"point": [
100,
600
],
"foreground": false,
"passed": true
},
{
"point": [
700,
600
],
"foreground": false,
"passed": true
},
{
"point": [
200,
200
],
"foreground": false,
"passed": true
}
],
"passed": true,
"modnetReferenceAgreement": {
"iou": 0.9850542728105918,
"alphaMAE": 0.005443826391281256,
"note": "Model-reference agreement, not human ground truth"
}
},
"runtime": {
"durationMs": 705.2900000028312,
"inferenceMs": 705.3149999976158,
"loadMs": 3490.719999998808,
"totalMs": 4196.034999996424,
"imagesPerSecond": 1.417806228427554,
"grade": "good",
"engine": "transformers.js",
"version": "4.3.2",
"actualDevice": "webgpu",
"actualPrecision": "q4f16",
"artifactRevision": "43ec3427514b8d9164eea02df93ca2f1b036bb7b"
}
},
{
"precision": "q4f16",
"device": "webgpu",
"status": "pass",
"quality": {
"platform": "browser",
"id": "portrait-repeat-4",
"source": "research/mvanet/browser-public-webgpu.json",
"runtime": {
"durationMs": 702.2299999967217,
"inferenceMs": 702.2449999973178,
"loadMs": 3490.719999998808
},
"probeAccuracy": 1,
"foregroundRatio": 0.2587576066068814,
"probes": [
{
"point": [
450,
120
],
"foreground": true,
"passed": true
},
{
"point": [
460,
210
],
"foreground": true,
"passed": true
},
{
"point": [
430,
350
],
"foreground": true,
"passed": true
},
{
"point": [
350,
420
],
"foreground": true,
"passed": true
},
{
"point": [
460,
520
],
"foreground": true,
"passed": true
},
{
"point": [
480,
630
],
"foreground": true,
"passed": true
},
{
"point": [
580,
470
],
"foreground": true,
"passed": true
},
{
"point": [
50,
350
],
"foreground": false,
"passed": true
},
{
"point": [
800,
350
],
"foreground": false,
"passed": true
},
{
"point": [
900,
100
],
"foreground": false,
"passed": true
},
{
"point": [
100,
600
],
"foreground": false,
"passed": true
},
{
"point": [
700,
600
],
"foreground": false,
"passed": true
},
{
"point": [
200,
200
],
"foreground": false,
"passed": true
}
],
"passed": true,
"modnetReferenceAgreement": {
"iou": 0.9850542728105918,
"alphaMAE": 0.005443826391281256,
"note": "Model-reference agreement, not human ground truth"
}
},
"runtime": {
"durationMs": 702.2299999967217,
"inferenceMs": 702.2449999973178,
"loadMs": 3490.719999998808,
"totalMs": 4192.964999996126,
"imagesPerSecond": 1.4240044428993008,
"grade": "good",
"engine": "transformers.js",
"version": "4.3.2",
"actualDevice": "webgpu",
"actualPrecision": "q4f16",
"artifactRevision": "43ec3427514b8d9164eea02df93ca2f1b036bb7b"
}
},
{
"precision": "q4f16",
"device": "webgpu",
"status": "pass",
"quality": {
"platform": "browser",
"id": "portrait-repeat-5",
"source": "research/mvanet/browser-public-webgpu.json",
"runtime": {
"durationMs": 705.7100000008941,
"inferenceMs": 705.734999999404,
"loadMs": 3490.719999998808
},
"probeAccuracy": 1,
"foregroundRatio": 0.2587576066068814,
"probes": [
{
"point": [
450,
120
],
"foreground": true,
"passed": true
},
{
"point": [
460,
210
],
"foreground": true,
"passed": true
},
{
"point": [
430,
350
],
"foreground": true,
"passed": true
},
{
"point": [
350,
420
],
"foreground": true,
"passed": true
},
{
"point": [
460,
520
],
"foreground": true,
"passed": true
},
{
"point": [
480,
630
],
"foreground": true,
"passed": true
},
{
"point": [
580,
470
],
"foreground": true,
"passed": true
},
{
"point": [
50,
350
],
"foreground": false,
"passed": true
},
{
"point": [
800,
350
],
"foreground": false,
"passed": true
},
{
"point": [
900,
100
],
"foreground": false,
"passed": true
},
{
"point": [
100,
600
],
"foreground": false,
"passed": true
},
{
"point": [
700,
600
],
"foreground": false,
"passed": true
},
{
"point": [
200,
200
],
"foreground": false,
"passed": true
}
],
"passed": true,
"modnetReferenceAgreement": {
"iou": 0.9850542728105918,
"alphaMAE": 0.005443826391281256,
"note": "Model-reference agreement, not human ground truth"
}
},
"runtime": {
"durationMs": 705.7100000008941,
"inferenceMs": 705.734999999404,
"loadMs": 3490.719999998808,
"totalMs": 4196.454999998212,
"imagesPerSecond": 1.4169624575808832,
"grade": "good",
"engine": "transformers.js",
"version": "4.3.2",
"actualDevice": "webgpu",
"actualPrecision": "q4f16",
"artifactRevision": "43ec3427514b8d9164eea02df93ca2f1b036bb7b"
}
},
{
"precision": "q4f16",
"device": "webgpu",
"status": "pass",
"quality": {
"platform": "browser",
"id": "dog",
"source": "research/mvanet/browser-public-dog.json",
"runtime": {
"durationMs": 739.2599999979138,
"inferenceMs": 739.375,
"loadMs": 3268.234999999404
},
"probeAccuracy": 1,
"foregroundRatio": 0.37315424610051995,
"probes": [
{
"point": [
200,
150
],
"foreground": true,
"passed": true
},
{
"point": [
350,
200
],
"foreground": true,
"passed": true
},
{
"point": [
500,
400
],
"foreground": true,
"passed": true
},
{
"point": [
0,
0
],
"foreground": false,
"passed": true
},
{
"point": [
690,
570
],
"foreground": false,
"passed": true
},
{
"point": [
680,
10
],
"foreground": false,
"passed": true
}
],
"passed": true
},
"runtime": {
"durationMs": 739.2599999979138,
"inferenceMs": 739.375,
"loadMs": 3268.234999999404,
"totalMs": 4007.609999999404,
"imagesPerSecond": 1.3524936601859678,
"grade": "good",
"engine": "transformers.js",
"version": "4.3.2",
"actualDevice": "webgpu",
"actualPrecision": "q4f16",
"artifactRevision": "43ec3427514b8d9164eea02df93ca2f1b036bb7b"
}
}
]