Body & Behavior Twins

Body & Behavior Twins

Intelligent behavior ultimately unfolds through bodies moving in the physical world. We extend the digital-twin paradigm beyond neural circuits to motor behavior and physical systems: learning probabilistic 3D pose estimators from multi-view video, reconstructing biomechanical trajectories with calibrated uncertainty, and modeling behavioral idiosyncrasies in both humans and animals. In parallel, we combine deep learning with physics-based forward models — using finite-element simulations for electrical impedance tomography (EIT) of the lung — to build hybrid models that respect known physical constraints while learning complex structure from data.

2025

Sara Rajaram, James R. Cotton, Fabian H. Sinz Similarity as Reward Alignment: Robust and Versatile Preference-based Reinforcement Learning arXiv
James R. Cotton, Fabian H. Sinz Biomechanical Reconstruction with Confidence Intervals from Multiview Markerless Motion Capture EMBC 2025 fully contributed paper
conference paper · body & behavior twins · arXiv

2024

Dominik Becker, Anita Just, Günter Hahn, Peter Herrmann, Leif Saager,Fabian Sinz RESIST: Remapping EIT Signals Using Implicit Spatially-Aware Transformer ML4H Symposium 2024
workshop paper · body & behavior twins ·
Pawel A. Pierzchlewicz, Caio da Silva, James Cotton, Fabian H. Sinz Platypose: Calibrated Zero-Shot Multi-Hypothesis 3D Human Motion Estimation arXiv

2023

Kacper Ksiezak, Rene Burghardt, Neda Shahidi*, Alexander Gail*, Fabian H. Sinz* Predicting choices in a dyadic foraging task using gated recurrent networks Complex Networks 2023 , equal contribution: *
extended abstract · body & behavior twins ·
Paweł A. Pierzchlewicz, Mohammad Bashiri, R. James Cotton, Fabian H. Sinz Optimizing MPJPE promotes miscalibration in multi-hypothesis human pose lifting ICLR - tiny paper
conference paper · body & behavior twins · openreview

2022

Paweł A. Pierzchlewicz, R. James Cotton, Mohammad Bashiri, Fabian H. Sinz Multi-hypothesis 3D human pose estimation metrics favor miscalibrated distributions arXiv