ArcFace and the alternatives: choosing a practical face pipeline

First separate four choices that are often confused

A detector locates faces. A backbone converts an aligned crop into features. A training objective shapes how that backbone learns. A runtime executes the trained graph. SCRFD, R50, ArcFace and ONNX Runtime therefore name different layers of the system; they are not four competing recognition products.

The same training objective can be used with several backbones and datasets. Likewise, two R50 checkpoints can behave differently because their training data, preprocessing or optimisation differ. A comparison labelled only “ArcFace versus AI model X” hides important variables. Record the exact weights, training provenance, alignment and evaluation setup before drawing a conclusion.

For CS Match the current combination is a five-keypoint SCRFD detector, an R50 ArcFace-family checkpoint and consistent 112 × 112 RGB recognition input. The displayed person crop is separate from that tensor. Choosing this combination is a product engineering decision; it does not imply that every model using the ArcFace loss has identical accuracy.

Understand checkpoint training and evaluation →Our 512-dimensional representation →

Recognition alternatives and what they change

FaceNet learns an embedding through triplets: an anchor should be closer to a same-person example than to a different-person example by a margin. Triplet selection is part of training. Its original research established a useful embedding-based approach; an arbitrary third-party “FaceNet” download need not reproduce the original model or results.

CosFace introduces an additive margin in cosine space after feature and weight normalization. ArcFace instead adds the margin to an angle before taking its cosine. Both aim to separate training identities more clearly, while still producing representations that can be compared at inference.

AdaFace adapts its training margin using feature-norm information as a proxy for image quality. MagFace also connects the learned representation to quality through its magnitude and a different training objective. These are credible candidates for difficult imagery, not properties that can be added to our fixed R50 by renaming its score.

A quality-aware model also has an integration consequence. If a method relies on the feature magnitude before normalization, storing only our unit-length vector discards that quantity. A proper evaluation must preserve the additional outputs the candidate method needs, rather than assuming all 512-value arrays are interchangeable.

FaceNet paper →CosFace paper →ArcFace paper →AdaFace paper →MagFace paper →

The evidence includes cases where an alternative wins

The AdaFace authors publish an R100 / WebFace4M comparison on IJB-C at FAR 0.01%. Their table reports 96.77% TAR for ArcFace, 96.86% for CosFace and 97.39% for AdaFace. The chart uses that particular comparison, with the authors’ attribution that other-method numbers come from the respective papers.

This is stronger evidence than mixing unrelated LFW scores from different training collections, but it is still author-reported evidence rather than our local reproduction. The common R100 and dataset labels do not establish that every training or inference detail was identical. These results cannot be assigned to our R50 or used as a measured CS Match error rate.

We therefore treat AdaFace as a serious candidate if a representative evaluation identifies low-quality recognition as the limiting stage. We would test the candidate’s complete preprocessing and runtime path. Its reference implementation, for example, expects BGR input whereas the InsightFace input convention used here is RGB. Feeding the right weights the wrong tensor is not a fair comparison.

● ● ●CS Match / Face AnalysisPublished benchmark

AdaFace author results and input convention →

Detector size is a separate quality–cost tradeoff

The SCRFD documentation reports the five-keypoint 0.5G, 2.5G and 10G variants on the Easy, Medium and Hard WIDER FACE splits using VGA evaluation. The larger variant has stronger published detection AP on these splits and a larger nominal computation budget. Our current detector belongs to the 2.5G five-keypoint family.

The chart is detection evidence, not identity-verification evidence. FLOPs describe a computation convention at the stated resolution; they are not the time a production request takes. Our image sizes, orientation passes, runtime and CPU scheduling change request cost. A larger detector is worth testing when missed small or difficult faces are the measured bottleneck.

A recognizer cannot recover a face that the detector missed. Conversely, replacing the detector does not automatically repair a poor embedding of a detected face. Keeping the stages separate lets us improve the actual failure rather than buying a larger network in the wrong part of the pipeline.

● ● ●CS Match / Face AnalysisPublished benchmark

SCRFD pretrained KPS results and evaluation conditions →Detection, alignment and rotation in our worker →

Fix orientation before judging the recognizer

A sideways face and a profile face are different problems. In-plane rotation changes where the eyes point in the image. Yaw turns the head away from the camera and can hide features. Rotating the photo can correct the first; it cannot reconstruct a cheek or eye that was never visible.

Our context crops use the eye line and the mouth direction to determine upright orientation, including the otherwise ambiguous 180-degree case. Standard analysis can refine unreliable orientation using quarter-turn passes for existing detections. Enhanced tilted-face detection searches eight orientations and merges repeated detections by image geometry before producing the report.

The recognition tensor is then built using the landmark alignment path. This is more useful than merely rotating a thumbnail with CSS: the model input and search handoff need the corrected image geometry too. Original-photo boxes remain in the original coordinate system so that the user can inspect which face was selected.

These steps address a real failure mode in the application. They do not guarantee a detection for every covered or tiny face, and eight passes cost more than a normal pass. Giving the user an explicit enhanced mode preserves a useful choice between additional work and ordinary interactive latency.

● ● ●CS Match / Face AnalysisTechnical schematic

Inspect the full image pipeline →

Self-hosted inference, managed APIs and retraining

A managed service can reduce the work of operating inference, but its usefulness depends on the outputs, limits, retention terms and controls it actually exposes. For our workflow, we need numbered group detections, a user-selected reference, explicit all-pairs results and a stable representation for indexed search. These are requirements to check, rather than assumptions about every vendor.

Self-hosting lets the application keep those stages explicit and version the preprocessing alongside the worker. It also places model rights, capacity planning, maintenance and evaluation on the operator. A runtime such as ONNX Runtime offers execution-provider choices, but switching CPU to GPU is a capacity decision to benchmark on the actual graph; it does not by itself establish better recognition accuracy.

Training from scratch offers control over the learning recipe at the cost of data preparation, compute and evaluation. Fine-tuning may help a well-defined domain, but can also move the embedding space away from the existing index. For an established photo-search collection, the cost of rebuilding and validating stored vectors belongs in that decision.

ONNX Runtime execution providers →Checkpoint provenance and reproduction →

Why our current combination is a reasonable quality choice

The quality argument for CS Match is the complete workflow. An established R50 representation is paired with face selection, consistent alignment, upright crops and the same vector conventions on both sides. The report shows the actual reference beside the source, preserves raw cosine similarity and distinguishes an unavailable computation from a valid low score.

In Multi compare, each photo’s vectors are reused for every pair. Results are sorted by similarity and can be reviewed without a forced one-to-one assignment. A user can inspect a crop at full size and continue to source research through Face Search. The system exposes enough of the input and calculation to catch common mistakes that a single unqualified “match” label would conceal.

For our interactive comparison and public-photo research use case, this is a defensible balance: strong published model-family evidence, controlled inference work and an inspectable result. It is our engineering judgement, not a universal ranking. A larger detector, a quality-adaptive recognizer or a specialised service may be the better choice for another capture regime.

• Correct person selection before scoring.
• Consistent geometry and representation across both photos.
• Reusable vectors instead of repeated inference for each pair.
• Source and reference images alongside the numerical evidence.
• Explicit limits and no invented end-to-end accuracy figure.

Try the all-pairs report →Understand raw and displayed scores →

What would justify changing the implementation

Start with the errors, not a model name. If the detector misses small faces, compare detector variants at a fixed input budget. If landmarks fail on rotated faces, test the orientation and alignment path. If valid aligned crops produce false or missed matches, evaluate alternative recognition checkpoints at the intended error tradeoff.

Use the same permitted image collection, identity split, operating-point procedure and hardware budget for each candidate. Report detection recall separately, invalid-vector counts separately, then verification errors and retrieval behaviour. Include median and tail latency, memory use and the cost of rebuilding the index. An improvement that breaks the response-time budget may need a different serving design.

Only after that comparison should we replace weights or introduce another model path. Version both the old and new spaces, rebuild the required vectors, re-evaluate thresholds and retain a rollback route. This is how we keep the current implementation reasonable and improve it when evidence supports a better one.

Practical evaluation and error limits →Training and benchmark methodology →