Body & Behavior Twins

Body & Behavior Twins

Intelligent behavior ultimately unfolds through bodies moving in the physical world. We extend the digital-twin paradigm beyond neural circuits to motor behavior and physical systems: learning probabilistic 3D pose estimators from multi-view video, reconstructing biomechanical trajectories with calibrated uncertainty, and modeling behavioral idiosyncrasies in both humans and animals. In parallel, we combine deep learning with physics-based forward models — using finite-element simulations for electrical impedance tomography (EIT) of the lung — to build hybrid models that respect known physical constraints while learning complex structure from data.

Contributors

Publications

2026

Sara Rajaram, James R. Cotton, Fabian H. Sinz Similarity as Reward Alignment: Robust and Versatile Preference-based Reinforcement Learning RLBRew 3 Workshop
Seth Donahue, Irina Djuraskovic, Kunal Shah, Fabian H. Sinz, Ross Chafetz, R. James Cotton Calibrated Uncertainty for Trustworthy Clinical Gait Analysis Using Probabilistic Multiview Markerless Motion Capture IEEE Transactions on Biomedical Engineering
journal paper · body & behavior twins · IEEE

2025

Sara Rajaram, James R. Cotton, Fabian H. Sinz Similarity as Reward Alignment: Robust and Versatile Preference-based Reinforcement Learning arXiv
James R. Cotton, Fabian H. Sinz Biomechanical Reconstruction with Confidence Intervals from Multiview Markerless Motion Capture EMBC 2025 fully contributed paper
conference paper · body & behavior twins · arXiv

2024

Dominik Becker, Anita Just, Günter Hahn, Peter Herrmann, Leif Saager,Fabian Sinz RESIST: Remapping EIT Signals Using Implicit Spatially-Aware Transformer ML4H Symposium 2024
workshop paper · body & behavior twins ·
Pawel A. Pierzchlewicz, Caio da Silva, James Cotton, Fabian H. Sinz Platypose: Calibrated Zero-Shot Multi-Hypothesis 3D Human Motion Estimation arXiv

2023

Kacper Ksiezak, Rene Burghardt, Neda Shahidi*, Alexander Gail*, Fabian H. Sinz* Predicting choices in a dyadic foraging task using gated recurrent networks Complex Networks 2023 , equal contribution: *
extended abstract · body & behavior twins ·
Paweł A. Pierzchlewicz, Mohammad Bashiri, R. James Cotton, Fabian H. Sinz Optimizing MPJPE promotes miscalibration in multi-hypothesis human pose lifting ICLR - tiny paper
conference paper · body & behavior twins · openreview

2022

Paweł A. Pierzchlewicz, R. James Cotton, Mohammad Bashiri, Fabian H. Sinz Multi-hypothesis 3D human pose estimation metrics favor miscalibrated distributions arXiv