[CVPR Workshop] Devis Tuia and Ronny Häensch: EarthVision

CVPR 2019

This episode is a live recording of our interview with Devis Tuia and Ronny Häensch at the CVPR 2019 conference. During the interview, they shared key product from their CVPR workshop"EarthVision", which offers software for for 3D model building, analysis, and visualization, with precise 3D models that can be quickly created and updated. Devis Tuia is professor at Wageningen University focuses his research on geospatial computer vision and served in the program committee at CVPR 2019. Ronny Häensch is a post-doctoral researcher with the Department of SAR Technology at DLR. His research focuses on machine learning, remote sensing, digital image processing and computer vision. He received his degree at the Berlin Institute of Technology. / Full Interview Transcripts / Wenli: Here we have Ronny Haensch from Technical University of Berlin. We also have Devis Tuia from University of Wageningen from Netherland coming here to share with us the workshop they just hosted called “Earthvision”. What is it about? Tell us a little more to our community who couldn’t attend CVPR. Ronny Haensch: Earthvision is mainly about the connection between computer vision and remote sensing. Remote sensing means earth observation from satellites over things. Most what we do is use computer to indicate what the conditional techniques using signal processing. Now, more and more computer vision also plays an important roles in analyzing the satellites data we collected. Wenli: How did you organize this workshop? Devis Tuia: This wasn’t the first time. We had the first preparation in 2015 in Boston, and 2017 in Hawaii. It was the recipe that we knew. And we put together a team of people who are willing to put some of their time on the workshop. Wenli: What kind of people did you select to co-organize this workshop? Devis Tuia: More people working in different areas of the field and evolve into technical activities. So people are already working at this interface between computer vision and remote sensing. We had Ronny for example, he is the chair of Image Analysis and Data Fusion Technical Committee (IADFTC) of data scientist and remote sensing scientists. He is a person quite central in the remote sensing community. We gathered some people very active in academia who organized the last year which is another large-scale, remote competition. The thing came together very naturally. Wenli: How was it today? Did you get to meet a lot of scientists around the world or in the US? What could junior scientists learn from this workshop today? Ronny Haensch:: There's still a lot to be done. Nice to hear from junior researchers, students, not everything's already solved. And it's really hard to pick a topic you can be successful. Wenli: What are some of the topics that you're still working on that is challenging? Ronny Haensch: Well, there are some traditional topics still around like areas where you want to know semantically, but you should transform into the image. There are more high-level things, building footprint detection, for example, counting any emails from UV images, there’re new data sources around with higher resolution, more feed aiders around. So there'll be all sorts of challenges of dealing with this huge amount of data, images that have a couple of gigabytes, for example. So it's very nice, there’re still those every core challenges around, even a young researcher can make an impact. Devis Tuia: Image sensing is not to an ending in the sense that there are some very important and big problems that can be tackled with that, and we had one of our keynote speakers, Danielle R. Wood. She showed up that there are this international goal for sustainable development goals, which is eradicate hunger, plan better cities to improve the quality of the air, of the water. And these are the things that we can observe from satellites from the air and model in order to help people who are taking decisions to take the right decision. Wenli: By that what do you mean? Assuming this will happen, or tell people this is happening at this location? Devis Tuia: Both. You can monitor what's going on with satellites, the atmosphere, the composition, and pollution in the water, anything that you can observe. What you observe is radiation, is energy. And then it will be able to make the link between this radiation and the actual thing you observed. And we have so much data now that computer vision is a game changer. Wenli: Yeah. But isn't the bottleneck for computer vision at this moment is data? Like the labeling and so many domains? Ronny Haensch: Labeling is a problem. We have a lot of data in different modalities from different sensors. What's even harder in remote sensing than the close-range computer vision applications is the labels. For remote sensing, it’s even harder to get the labels because in many areas, you can’t really observe what’s there. Other things are how to measure, for example, soil moisture, you can’t just go there and watch how wet it is. So it's really hard to measure that in order to get really ground truth or reference data. So this might be more true for remote sensing applications than for very close-range computer vision applications. Devis Tuia: There’re new vision concepts like weakly or semi-supervision approaches. Things are applied really well here. It’s an uncharted territory. Wenli: Computer vision is also receiving so much attention lately in the last few years. And CVPR is growing tremendously as you have attended a few CVPRs, last year there was like 9k people for sure, right? What kind of breakthrough that you observed in the field of your study? Ronny Haensch: One of the biggest breakthroughs over the last years was that the Chronicles program made satellite data from specific satellite combination really accessible to everybody for free. So now we have reader images, optical images optical range, that's just for free for everybody. This has enabled a lot of different techniques that have not been really possible before, including deep learning techniques, and also computer vision techniques. Devis Tuia: This one has taken longer time efforts like NASA standard set or any image for a long time, and very recent thing of super high resolution like launching hundreds of satellites to make a daily coverage of the earth. So the amount of data is really taking the elevator. And I think the breakthrough for people living at this interface is really here, in the sense that the data being recorded is probably not being used in the way it should be. Wenli: I know that you're both working in academia. Do you see any business applications in the short term for the areas you're working in? Ronny Haensch: Certainly, there are a couple of companies more focusing on remote sensing. There’re a couple of companies, for example, observing air quality based one satellite data. Their companies are trying to detect illegal racist signs, things like that. So there are more and more companies using the remote sensing data in order to either improve their work in anyway or solely based on remote sensing data. Devis Tuia: I’ve seen business cases popping up all the time. If you're a student being hired by insurance companies, you want to monitor crops in an objective way in order to decide whether there is a farmer after storm. I see remote sensing being used not only in additional fields, but really becoming democratic. It’s going to be like this more and more. Wenli: In the recent few years that you attended CVPR, what are some of the differences that you noticed in each year? Ronny Haensch: It's my first CVPR for me. Devis Tuia: This is my fourth. It’s become very big, growing a lot. Industry is engaging more and more. So. This position is more and more incredible. And it's exciting for deep learning, of course, also the community has been growing more and more, with all these new results and new achievements. It’s really good to be here. Wenli: Each year in CVPR, you come here to meet people and it’s just exciting to share all the new things that people have been working on. And then you go back to the university to solve your problems. What are some of the differences of working in the research teams in Europe compared to the research teams you’ve met in the US? Ronny Haensch: One thing I saw a lot in the US teams is that the students and postdocs are really closely collaborating on the CVPR publications, for example. The teams are a lot bigger. While in Europe. I mean, there's also cooperation going on but usually with two or three people collaborating. Very often there’s only one person, supervised by somebody, of course, but still just one person. So that might be one of the differences Devis Tuia: From another 10 years seems that there is a lot of pushing in the research in America, a lot of funding being injected in the system. That's maybe just the view from the outside. The industry is very much present, which is probably something that is also happening in Europe, but a little bit slower. In Europe, it seems like comparing to America, things are not working very well. That's the difference. Wenli: What would be some of the challenges that you will be working on for the next year? Devis Tuia: Using computer vision to change the world. For me, it’s AI for good. It’s something really important for me try to make a difference using technology. Wenli: We mentioned some of the challenges like how to label the data, for example. What are some challenges that you're trying to solve in the next coming year? Ronny Haensch: One challenge is that the remote sensing currently of two worlds are barely connected. One world is the machine learning computer vision group that is looking at years of image data and apply machine learning out of the box to solve the problems. And the other world is saying, don't do any learning-based approaches which are all equal to traditional ways where we really modeled the physical process of measurement and linked it to our target. So they have a physical model to try to invert. Those two worlds are totally. One of the challenges will be to connect those two worlds, because we do know a lot about how the data is generated, which physical process really leads to the measurement, and this prior knowledge should be exploited and included into the learning bases. So those two worlds need to grow together and benefit from each other instead of go in separate ways. And this is happening more and more, slowly though, but it's happening. This is the thing that will be the challenge, at least for me for the next years. Wenli: That's great. How about you? Devis Tuia: I think involving a machine learning AI vision agenda in the space agencies so that the next generation of satellites maybe built in a way as we also need to make the impact that we want to do is a challenge for all of us in the community of remote sensing. Wenli: Just out of curiosity, do you have privacy issues when you are using those satellite images? Ronny Haensch: In Germany, yes. There’re some imagery you're not allowed to show to others if the resolution is too high. Then we need to provide a list of authorized people to the data provider. But apart from that, for research, you can use them. Devis Tuia: There’re some images you can access for free, for some you need to pay for them. Personally I never ran into any problems. Wenli: That’s something that I'm always curious about remote sensing. Devis Tuia: We don't use images with such high resolution which may become a problem. Probably with the drones is probably starting to generate high-resolution images, and it might become a problem. Wenli: Thank you so much for you to stop by to share with us your experiences. Ronny Haensch: Thank you for having us. Devis Tuia: Thank you.