By using Recorded Future in conjunction with traditional ethnographic techniques, it is possible for us to significantly increase our accuracy in forecasting international and diplomatic crisis. This is due to the recurring nature of certain patterns present in these crises. Take for example the proliferation crises with North Korea or Iran. Recorded Future’s web intelligence platform helps us identify those patterns, ethnography helps us understand them. Combining these two gives us an edge we otherwise would not have. In this post I will explain how this methodology works.
One Way to Forecast International Crisis
First, we need a way to objectively measure the existence of a crisis. But how do we measure the existence of more or less tension between North Korea and the international community, or between China and Japan? Political and diplomatic tensions, I would like to posit, are reflected in negative sentiment in the news. Sentiment analysis can effectively reveal trends when applied to large amounts of text with a similar writing style, like news reporting. Sentiment in the online news can be easily and quickly quantified using open source intelligence from Recorded Future.
The following timeline shows a general example of how Recorded Future organizes information from the web and via its algorithms determines whether the reference offers negative or positive sentiment. Below we see red and green dots, the former representing negative sentiment, the later positive.
Click image for larger view
Second, we need to be able to view the rise and fall of these political tensions over a given time frame in the past. If we organize the negative sentiment from Recorded Future over time in the form of a line chart, we get something that looks like this.
Now that we have a way to objectively measure tensions, we can turn our attention to the next question: How do we make an accurate forecast? The key here is to look for recurring patterns in the past concerning a particular crises. Let’s use the example of North Korea.
Every green dot along the way represents a momentary reversal of tensions. Things were bad, but then they started getting better. What was happening at all those points in time? We identify the common characteristics to all of those events in the past, by again using web intelligence to look at what was happening in different subtypes of data, such as the number of comments to mainstream news articles, the number of blog posts, and the frequency of Google search trends. Finally, we ask what is happening in real-time data. If circumstances in the subtypes of data are the same as they were in the past, a signal is made.
Click image for larger view
This procedure requires us to quickly sift through very large amounts of data, too much for humans to manage. But it can be done with algorithms. At Ethnographic Edge we have developed an algorithm called Social Risk Analysis (SRA) because it looks at relations between social phenomena (behavior) and quantifies risk as a percentage of possibility. However, algorithms often fail to properly contextualize random or unexpected events. Human perception is necessary to make the connections that machines cannot possibly be programmed to look for in advance.
Ethnography is a methodology used to uncover hidden dynamics. By contacting a network of local informants and interlacing their opinions with information from other sources, such as academia and think tanks, we get an ethnographic review. Through this ethnographic review, we get information we cannot get via web intelligence.
For example, we learn the Senkaku/Diaoyu Island dispute is used by both China and Japan principally for domestic political purposes – a point which is rarely highlighted in the international media. We can then use this ethnographic review to confirm the signals created by the Social Risk Analysis. This combined approach increases the accuracy rate. There is an increased reliance on automated intelligence by analysts. The methodology put forth here builds on that trend by applying algorithms to social phenomena and then integrating them with human analysis.
3 Benefits of Ethnography
Does this work? Yes, it is possible to obtain a statistically relevant increase in our accuracy this way. What are the benefits? I believe there are three ways in which this may create value.
First, this type of semi-automated analysis helps avoid the emotionality and vagueness we are subjected to through forecasts made by pundits on the mainstream media. Second, it allows us to avoid informing our conclusions with a political agenda, as is the case with forecasts coming from the think tank and policy making environments. These two benefits improve the credibility and accuracy of forecasting for international crises.
Third, developments concerning international crises can be correlated with the fluctuation in price of particular economic assets.
I leave you with the specific example of the political crisis that unfolded last fall in Thailand, in which correctly forecasting an increase in political tensions preceded a sharp fall in value of the Thai Stock Exchange. The trend of political tensions, when combined with ethnographic factors, has predictive value for forecasting period of major price volatility.
Click image for larger view
The post Is it Possible to Use Web Intelligence to Forecast International Crisis? appeared first on Recorded Future.
