Beyond Family Labels: A Taxonomic Ornstein–Uhlenbeck Prior for Avian 3D Shape Recovery

1Chung-Ang University   2GenGenAI   3Konkuk University
NeurIPS 2026 (Oral)

*Equal contribution   †Corresponding author
Animated qualitative comparison on eight CUB-200 test-split species absent from the 3D fine-tune set: AniMer+ keypoints (red) versus ours (blue) moving toward human-labeled ground truth (green), with PCK@0.1 and mean keypoint error

Keypoints beyond the 17 fine-tuning species. On CUB-200 test-split species absent from the 3D fine-tune set, AniMer+ (red ×) misplaces keypoints; fine-tuning with our taxonomic OU prior (blue ×) moves the keypoints toward the human-labeled ground truth (green •), raising PCK@0.1 and lowering mean keypoint error on each case shown.

Abstract

Recovering 3D pose and shape of birds from single images is bottlenecked by the long tail of avian diversity: out of ~10,000 species, only a few dozen have 3D-labeled training data, and in-the-wild test sets routinely contain species never seen at training. Generalizing across clades is therefore a central problem. The current state of the art, AniMer+, mitigates it with a family-level contrastive loss that conveys only a binary same-family indicator per pair and discards the continuous structure of the species tree.

We propose a taxonomic Ornstein–Uhlenbeck prior on the shape code of the AVES parametric bird mesh model: a multivariate Gaussian whose covariance decays exponentially with Linnaean rank distance, with a learnable selection strength and noise scale. The prior generalizes family contrastive into a continuous, generative form, exposing the graded cross- and within-family similarity that a binary indicator cannot, and brings continuous comparative phylogenetics in as a biology-grounded inductive bias for the long tail.

Our method improves over AniMer+ on both metrics of the unseen-species CowBird benchmark and on CtrlAVES3D, while remaining comparable on seen-species CUB. A permutation ablation isolates taxonomic structure from generic Gaussian smoothing, and a rank-granularity ablation identifies cross-order distinctions (the structure beyond a family-vs-not indicator) as the dominant source of the unseen-species gain.

Overview

The 17 training species of CtrlAVES3D on the Linnaean tree, colored by order, with the lowest shared rank from same family up to outgroup

The 17 training species on the taxonomic tree. AniMer+ only asks whether two species join below the dashed red line (same family). Our prior uses every level: the closer two species are, the more strongly their shapes are tied.

Click the tree to see how it becomes the distance matrix.

Method

Method at a glance: (a) AniMer+ encoder and AVES shape head produce a shape code; (b) a taxonomic OU prior with learned alpha, sigma, mu over a fixed Linnaean rank-distance matrix gives a Gaussian NLL on shape codes; (c) the OU loss replaces the family-level contrastive term in the training objective

Click the figure for a panel-by-panel walkthrough.

Overview. We keep the AniMer+ network (blue) and change only its training loss (orange). Closely related species tend to have similar body shapes, and the similarity fades as the relationship becomes more distant. Comparative biology models exactly this with an Ornstein–Uhlenbeck (OU) process: a trait evolving along the species tree, whose correlation between two species decays exponentially with their tree distance. We place that model as a Gaussian prior on the 15-dimensional AVES shape code β, with the distance read from standard Linnaean taxonomy, and use its negative log-likelihood in place of AniMer+’s binary family contrastive loss. The network, and its cost at test time, stay the same; the only new parameters are the two scalars α, σ and the mean μ.

Step by step

Click any step for the full explanation.

1 Input

Linnaean rank distance

A fixed 17×17 matrix of integer ranks read off standard taxonomy: 0 same species, 1 same family, 2 same order, 4–6 across orders.

\(d_{\mathrm{tax}}(i,j) \in \{0,1,2,4,5,6\}\)
2 Prior

OU covariance over species

Correlation between two species’ shape codes decays exponentially with their tree distance.

\(\Sigma_{ij} = \frac{\sigma^2}{2\alpha}\, e^{-\alpha\, d_{ij}}\)
3 Loss

Gaussian NLL on shape codes

Score each mini-batch of shape codes under the prior, with Σ indexed by their species labels.

\(\mathcal{L}_{\mathrm{OU}} = -\log \mathcal{N}(\boldsymbol\beta;\, \boldsymbol\mu,\, \Sigma)\)
4 Relation

Family contrastive is a corner case

In the α → ∞ limit the kernel reduces to family blocks, a structural analogue of AniMer+’s binary same-family view. At α ≈ 1, where training keeps it, the prior stays graded.

\(e^{-\alpha d}:\ 0.37,\ 0.14,\ 0.02,\ \dots\)
5 Training

Drop-in replacement in AniMer+

Swap the contrastive term for the OU term. Same network at test time, so no extra cost.

\(\mathcal{L}_{\mathrm{total}} = \mathcal{L}_{\mathrm{AVES}} + \lambda_{\mathrm{OU}}\,\mathcal{L}_{\mathrm{OU}}\)

How it works: a four-bird toy example

Two sparrows (one family) and two warblers (another family), all in one order. Drag α to see the covariance change; click a card for details.

ANIMER+

Tree cut at family → 1 or 0

discarded above

Same family = 1, anything else = 0.

OURS

Whole tree → covariance

1.00

Similarity decays with tree distance: e−αd.

OURS

NLL pulls codes toward relatives

shape space ? unseen species borrows from relatives before after pull = Σ

Strong pull within a family, weak within an order: family neighborhoods tighten, while the tree-like layout from pre-training is kept.

Results

Keypoints move toward the ground truth

Qualitative comparison on two CUB test images and two CowBird test instances: input with ground-truth keypoints, retrained AniMer+ baseline, and ours

CUB (top) and CowBird (bottom). Green: ground truth · red ×: AniMer+ · blue ×: ours.

The gain is on unseen species

MethodCUB seenCowBird unseenCtrlAVES3D seen
AUCPCK@0.1AUCPCK@0.1AUC
AVES79.7025.2046.105.6085.90
HMR2.087.7050.0059.1012.2091.60
AniMer-A†91.9074.8065.7019.4092.90
AniMer+89.5059.4059.3016.4093.00
Ours89.30±0.0257.84±0.0960.80±0.1018.86±0.2793.05±0.01

Bold: best among unified bird+mammal models. Ours: mean ± std over 5 seeds. †Bird-only specialist, not a like-for-like comparison.

The gain comes from the real tree

AniMer+ 16.40 shuffled tree 18.49 real tree 20.25

CowBird PCK@0.1. Same loss, but species labels shuffled in the distance matrix → the gain shrinks. Both OU runs use the ablation setting (λOU = 10−4); the AniMer+ bar is its paper-reported value.

Ranks above the family carry most of it

family only +0.60 + order +0.71 full tree +1.76

CowBird PCK@0.1 gain over the shuffled tree. Family membership alone, the same information AniMer+ uses, recovers only about a third of it.

BibTeX

@inproceedings{jeon2026beyond,
  title     = {Beyond Family Labels: A Taxonomic Ornstein--Uhlenbeck Prior for Avian 3D Shape Recovery},
  author    = {Jeon, Yunchan and Kim, SooGon and Kim, Seungwook and Lee, CheolWon and Lee, Jongmin},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026}
}