The main trick: IMU sensor fusion The biggest bang-for-buck stabilizer is adding a cheap IMU (inertial measurement unit) to your depth camera. When retail lighting screws up visual tracking—harsh spotlights mixed with fluorescents—the IMU keeps giving orientation and acceleration data. Even when your camera loses features in bright patches or dark corners, the system doesn't lose track. A decent IMU runs $10-30, way cheaper than upgrading cameras. There's also a software trick: swap traditional ORB features for deep learning-based extractors like SuperPoint with LightGlue matching. Purely software, handles lighting changes way better. Hardware that holds up -Azure Kinect DK (~$400, hard to find) Time-of-Flight infrared, cleanest depth data Best for full-body tracking, handles multiple units without interference Downsides: expensive, out of stock, struggles with direct sunlight -Intel RealSense D455 (~$240) Stereo vision with IR structured light Affordable and available, runs at 90fps Noisier than Kinect but usable with filtering Pair with IMU (D435i has one built in) -Intel RealSense L515 (~$350) LiDAR-based, super compact Depth quality varies with lighting Other helpers Image preprocessing (Zero-DCE, EnlightenGAN) boosts dim areas—all software. Some use two cheaper sensors with fusion algorithms instead of one expensive one. Adaptive feature extraction (CLAHE) helps with mixed bright/shadow areas. Budget recommendation RealSense D455 or D435i + SuperPoint/LightGlue + maybe image preprocessing on Jetson Nano. Around $250-350 total versus $400+ for Kinect. Reliable enough for try-on mirrors with typical mixed retail lighting.
A simple trick that helped a lot was locking exposure and white balance after an initial calibration, then running a quick illumination normalization step so the tracker sees consistent contrast even when lighting flickers or spotlights blow out highlights. For pilots, the most reliable setup was an RGB camera paired with a commodity depth sensor, using depth to stabilize pose and scale when the RGB stream got noisy. That combo handled glossy floors and uneven lighting better than RGB only.