The M2 Holo Lab project enables users to interact with 3D models using only a standard laptop webcam and a browser. The system utilizes MediaPipe for hand tracking, Three.js for rendering, and cannon-es for physics, all running locally on the user's computer.
To overcome the lack of depth perception and tracking jitter inherent in standard webcams, the developer implemented custom calibration, smoothing algorithms, and specific thresholds for gestures. The system uses visual cues, such as highlighting objects before interaction, to improve precision.
Performance optimizations included decoupling the tracking and rendering processes to increase frame rates. The developer also addressed rendering bugs on Apple hardware by adjusting shader logic and depth comparisons to prevent visual artifacts.
The application features modules for mechanical engineering, chemistry, and anatomy, incorporating real-world geometry for molecular structures and open-source organ datasets. A perspective-shifting mode mimics a holographic effect by adjusting the view based on head tracking.
Currently in closed beta, the project is being positioned as an educational tool for classrooms. Interested educators can apply for access through the official website, m2holo.com.
