Immediately following the Russian invasion of Ukraine on February 25, this man has continuously informed the world about the situation in Ukraine, using 3D processing technology and satellite images. He is Professor Hidenori Watanabe of the University of Tokyo Graduate School, working to create comprehensible digital maps by digitally mapping records of disasters and wars. Amid the ongoing tension in Ukraine, Watanabe talked about how he updates the digital maps and continues postings on Twitter (now X).

(#1 of a two-part series. Continues on #2 )

Digital Maps Made by Geo-Referencing Method

– How do you make the “Satellite Images Map of Ukraine,” the digital map of Ukraine that you post on Twitter?

Watanabe: Using Bing Maps, Microsoft’s web mapping service, as the base, we precisely attach high-resolution images provided by private satellite imagery companies. This method is called geo-referencing.

We mostly use images from Maxar Technologies of the U.S., but we also use satellite images from several other companies to create our digital maps. To display the maps, we use the open-source Cesium platform. This is a collaborative project with Professor Taichi Furuhashi of Aoyama Gakuin University.

– Why do you turn them into digital maps? How is it different from just posting satellite images?

Watanabe: You often see pictures of black smoke rising across a city or blown-up buildings on TV news. But that only conveys one-dimensional impression of a “terrible damage,” and the background or the context may get lost. To fill in the gaps, we turn them into digital maps.

“Places of Attack” Indicate Russian Military Moves

For example, there is this March 19 satellite image of Mariupol. The black part in the middle is the industrial zone. So, it’s likely to be an attack target. Yet, the orange places where there is fire are the urban areas around the industrial zone.

When considering why the urban areas were attacked, it’s helpful when you know about the surrounding environment, instead of just having the satellite image.

For example, we have this image of Chernihiv, about 120 km northeast of Ukraine’s capital, Kyiv. When you overlay this image on the digital map, you can see the attacked areas are along a broad highway.

When you look at a wider area, you can guess the Russian troops are progressing by attacking the urban areas along the highway. Checking on the digital map, you can see Chernihiv is located along the highway leading to Kyiv. So, you could say the Russian troops are trying to take control of surrounding cities like Chernihiv to enclose Kyiv.

By placing multiple satellite images on the digital map, you can make these observations and assumptions.

Human Eyes Identify the Location of Satellite Images

– How do you identify the locations of the satellite images? Do the images include location data?

Watanabe: No, basically, the distributed satellite images don’t have geo tags attached. So, we need to compare each one with the map and find the same objects to determine the location.

And unlike regular maps, satellite images don’t always have the north side on top. Especially the images from Maxar Technologies, which we use a lot, are already trimmed, probably to look good for media use. So, we turn the base map round and round to find the spot with the same object captured.

It’s especially hard with a zoomed-in satellite image for a narrow range. It may take half an hour to an hour until we can fix it on the map.

Shadows Provide a Big Clue for Orientation

To identify a place, the direction of the shadows is a big clue. Satellites with a sun-synchronous orbit, like those of Maxar Technologies, usually take the images at southing time, so the shadows would be cast toward true north. Using the direction of the shadows in the picture, we can guess the orientation of the satellite image.

For the image of the destroyed shopping mall in Mariupol, we determined the place by geographical assumptions. We guessed that, like in Japan, large shopping malls would be in the suburbs rather than in urban areas. Therefore, we decided to search out-of-town areas instead of the residential areas. And also using the direction of shadows I mentioned earlier, we found the place.

With mere guesswork, we may never find the place. So, I believe this is where I, Professor Furuhashi, and others can make the most of our expertise and experience.

– Aren’t there technologies to determine the location using AI?

Watanabe: Automatic identification itself is probably difficult for the current AI technology. Moreover, AI can’t contemplate by overlaying satellite images on digital maps, understand what’s going on, and communicate that to other people. Only humans can do that.

For example, when you overlay the satellite image of the destroyed apartment complexes in Mariupol on the digital map, you can see they are built around a school. In Japan, you also often see apartments built around a school. So, we superimpose our lives in Japan with what is happening in distant Ukraine. It provides the opportunity to think about it as your own issue. If we automate the task, such chances will be lost.

Little Fear of Being Misled by Fake Satellite Images

– How do you guarantee the credibility of the satellite imagery information?

Watanabe: In an age with such advanced digital technologies, it must be easy to create fake satellite images. In fact, AI technologies for creating realistic satellite images have been released. So, it’s not impossible to fabricate.

However, the feasibility is low for multiple satellite companies to conspire in an elaborate scheme to distribute satellite images containing the same fake information. If a company is involved in such a scheme even once, no one will trust their satellite imagery. We’re in an age where it’s easy to tell lies, but you get immediately exposed. So, no company would risk distributing fake images.

– Nowadays, there are countless private satellites flying around. Unlike the times when military satellite images were only available to governments and militaries, they are shown real time on mass media and social media. We could say you can’t really lie or disguise.

Watanabe: That’s right. This really shows the social situation that we live in in 2022. Satellite imagery companies not only distribute images to the mass media in various countries but also post on their websites and social media accounts. Also, the conditions for use are made clear. Anyone can use them by following the conditions. The method of using these publicly available material and data to research facts is called open-source intelligence (OSINT). Our activities are also like OSINT.

For example, in one satellite image, you can see an object considered to be the superyacht owned by the Russian oligarch, who has close ties with the Putin administration. This image was captured by Planet and offered with the Creative Commons license which allows reproduction.

This superyacht is anchored at the solitary island of Seychelles in the Indian Ocean, far removed from Russia. The oligarch has been criticized for cooperating with Putin’s military aggression, and sanctions have been announced, including U.S. assets freeze and banning of his company business.

For these reasons, the owner may be anchoring the superyacht in the Seychelles to avoid public eyes. Yet, it’s easy to find for a satellite that can capture every surface of the earth. This is one example of what we can do with OSINT.

Matter-of-Fact Reporting of Local Damages

– Since private satellite imagery companies are mostly American or European firms, could it be possible that pro-Ukrainian information is being distributed?

Watanabe: Well, it could be possible that information “inconvenient” for Ukraine is excluded. However, it’s certain that in Mariupol, there is mass destruction in the urban areas, where ordinary citizens live. This fact is unbiased; there are no “sides” to this information.

We create and publish the digital maps to inform only the real local situation, of the damage caused on people and cities. It’s not according to some ideology.

The posted digital maps also include images of Russian helicopters in flames after Ukrainian counterattack in the Russian-occupied airbase.

The war situation is still unclear, and in the future, we may receive images inconvenient for the Ukrainian military. In such cases too, we will not make any selections, but put them on the maps and publish them in a matter-of-fact manner.

I’m Here to Show the Fact in a Simple and Comprehensible Way

– The published digital maps have no commentaries or remarks added, with only the minimum necessary captions. Is this also because you want to “convey only the fact”?

Watanabe: I’m an information design researcher and not an expert on warfare or international politics. My job is to convey the fact in a simple and comprehensible way. So, based on the caption of the original article, we only provide minimal information, such as the date, place, and subject.

Of course, we could add comments like “it’s tragic” or “unforgivable.” But such negative emotions will get amplified online and create new conflicts.

I don’t mean to deny the people suffering damages in Ukraine from taking pictures and posting them along with messages like “the Russian forces are unforgivable.” But I think it’s wrong for me, far removed, to utter remarks from a safe distance.

As I’ve said earlier, I’m here to convey the fact in a simple and comprehensible way. Beyond that, it’s up to the person who sees the map on how they think and act.

Through our digital maps, of course, and through interview articles like this, I hope more people will feel that what’s happening in Ukraine is not someone else’s business, but also relates to us living in Japan.

Continues on #2

◆ Digital maps created by Hidenori Watanabe et al.

・Satellite Images Map of Ukraine
・3D data & 360 panoramas of Ukraine
・Satellite Images of Mariupol
・Satellite Images of Around Irpin
・Satellite Images of Chernihiv

This is a translation of the Japanese article published on March 31, 2022 on Bunshun Online.