BMVC 2020 - Real-time screen reading: reducing domain shift for one-shot learning

BMVC 2020

A demo of our system for reading digital meter screens in real-time using a mobile phone. The video demonstrates the use on a blood glucose monitor. The system can be applied to any digital meter screen and requires only a single annotated image for training. Abstract: Many digital meters such as those used for home health (e.g. blood pressure meters) or meters monitoring industrial equipment do not contain wireless connectivity. Hence, connecting these devices to phone tracking apps or control centres either requires cumbersome manual transcription or is not plausible due to costs. Our motivation is to cheaply retro-fit these types of meters with ‘smart’ data transfer capabilities using a mobile phone app and limited training data. We demonstrate how one can use single training images of meter screens to build efficient custom meter readers targeted to chosen devices. To this end, we build a CNN based system which runs in real-time on mobile device with very high read accuracy (close to 100%). Our contributions include (i) introduction of an exciting new application domain, (ii) a method of training from purely synthetic data by reducing domain shift using a surprisingly simple approach which unlike adversarial training based methods does not even require unlabelled data; (iii) a highly accurate system for parsing digital meter screens and (iv) release of a new screen reading dataset. The system, although trained solely on synthetic data, transfers very well to the real-world. Our method of screen detection and text recognition also improves over the state of the art on our dataset.