To support the International Brain Laboratory’s mission to understand neural dynamics underlying decision-making, this project documents and refines a robust, standardized video analysis pipeline for tracking head-fixed mouse behavior during cognitive tasks. As neural dynamics control movements, and movements in turn produce sensory input that modulates neural dynamics, detailed behavioral monitoring is essential for interpreting brain-wide neural data. The pipeline successfully tracks pupils, paws, tongue, and nose across hundreds of heterogeneous videos from a dozen experimental labs, achieving high accuracy despite significant variance in camera placement, lighting conditions, and animal appearance. This standardized approach provides the foundation necessary for relating neural activity to behavior at scale.

The pipeline processes video data from a multi-camera setup (two side cameras and one body camera) with specialized workflows including markerless pose estimation with Lightning Pose and motion energy computation in pre-defined regions of interest. Key technical innovations include a novel pose estimation architecture and unsupervised losses, ensemble kalman smoother post-processor, comprehensive quality control metrics for both raw video and tracking outputs, and automated infrastructure for processing hundreds of datasets at scale. This technical framework, openly available through the IBL video repository, provides the neuroscience community with a scalable, reproducible solution for high-throughput behavioral analysis in large-scale studies.