For edge inference procurement or evaluation, define the task and metrics to compare. Save the model file, input dimensions, batch size, numerical precision, runtime and hardware configuration with the results. The same model name alone does not prove that execution is identical. Version the converted file and its configuration too.
Define the input and timing boundaries
Use fixed samples and agreed preprocessing and output processing rules. Measure model execution, data copying, video decoding and application latency separately. State the start and end events and synchronization method. Record warm-up and measured samples. The execution time of one model operation does not represent total application latency.
Record shared resources and concurrent workloads
State simultaneous video streams, transcoding, recording and other tasks. Record CPU, memory and accelerator load. ONNX Runtime threading documentation describes execution modes and thread configuration. Measure the actual configuration on the target device under the intended workload. More threads do not constitute a promise of faster delivery.
Deliver results with their applicable conditions
Check outputs using the same samples and agreed quality metrics, then compare latency distributions, throughput and resource records. Report metric definitions, sample coverage and exceptions. Do not extend one model execution result to whole-device concurrency, long-term stability or other inputs.