Using AI to find financial opportunity in chaos

ISE Magazine August 2019 Volume: 51 Number: 8

By Joseph Byrum

https://www.iise.org/iemagazine/2019-08/html/byrum/byrum.html

Financial markets are inherently chaotic. For more than a century, the frenzy took physical form as orders were shouted from the trading pits of the major exchanges. Transactions were completed with hand signals that al-lowed for rapid-fire deal-making in a process where speed was everything.

Today, the majority of trades are negotiated from behind a computer screen in a quiet office building, often without human intervention. Nonetheless, what’s happening remains as fast-paced as it is unpredictable.

A good buy can still turn into a loser in a matter of moments. If the system were fully predictable in a way that could be replicated, an algorithm could guarantee the success of every transaction. Not only would this magic algorithm achieve “alpha” by beating the market in the long and short terms, losing would also be impossible. Such an algorithm would ensure high returns with zero risk, which is the sort of a claim that can only be made by charlatans or the extremely lucky.

Contrast financial markets’ ups and downs with the predictability of a factory, where items are manufactured in an orderly, step-by-step process. Factories are con-trolled environments specifically designed to be free from randomness and variation to the greatest extent possible. Luck is not a factor in a controlled environment.

Finance is anything but controlled, though its complexity is traditionally tamed with statistics. One can measure the various performance characteristics of a set of companies, properties or commodities. By comparing figures from a set of companies, an investor can spot the outliers, firms that, by the numbers, should be per-forming better on the market and might represent a bargain. This works because a set of common criteria can be applied to compare across wholly different fields. To take a simple example, you can compare the earnings per share of a media company like Disney against that of an airplane manufacturer like Boeing and decide that one makes for a better investment than the other.

What AI does differently

Artificial intelligence (AI) has the potential to change everything by minimizing the impact of luck. AI brings causal analysis to the table, which when implemented properly has the potential to replace luck with insight, at least to some measurable degree.

To do this right, the AI system must go beyond simple statistical analysis to fashion a more complete picture of business performance that takes into account the inherent market complexity and individual characteristics of each particular company of interest. It would build models and conduct simulations far more complex and complete than has ever been possible.

This approach borrows from the philosophy of Wilhelm Windelband, the 19th century follower of Immanuel Kant, who laid out two ways of describing reality. The first, the “nomothetic,” seeks to find what different items have in common so that they can be analyzed and compared. This technique relies upon generalization, which naturally lends itself to statistical analysis. The second method, the “idiographic” or “idiosyncratic,” examines each item as an individual within a specific context. Instead of statistics, idiosyncratic analysis has relied on pen and paper. Until now.

Mathematics is the ideal example of a nomothetic discipline. Numbers are abstractions, generalizations that are everywhere and always the same. Two plus two is always equal to four. When we need to apply mathematics to the real world, we can use modern statistical theory, which ac-counts for what appears to be random variation through the concept of population. All members of a population share a set of general properties; the larger the population, the more exact the estimates that statistics can make of the population’s properties.

Population models and their generalizations are less useful in describing things that are inherently unique. For instance, statistics and population models are less accurate in discussing history. Generalizations can only provide so much insight into the events of a given day or a given year, as these are populations of one. A full understanding of what happened on an important date can’t be reduced to statistics or numbers.

Mathematical abstractions also can fall short when applied to messy, chaotic circumstances. Say you’ve decided to grow corn crops on a 1-acre field. You plant seeds, which are known to have an average yield of 176 bushels per acre. Do you end up with 176 bushels of corn at harvest time? Maybe. It depends on the weather, how hungry the in-sects happen to be, whether there are any disease outbreaks, the prevalence of weeds, soil quality levels and hundreds of other individual factors that apply not just from one field to the next, but from one part of one field to another part of the same field. Simple statistical analysis is not the best way to deliver results.

So it is in finance, where the urge to generalize business performance is strong and is often reduced to a set of basic statistical calculations. While appropriate statistical models can offer a great deal of insight that can only go so far. As the standard financial disclaimer goes, past performance is no guarantee of future results. What AI can do differently is create simulations based on a population of one to pro-vide information on present and future performance under present and expected conditions, rather than just through a statistical analysis that extrapolates from past performance.

Markets are not the same from one day to the next, from one year to the next or from one decade to the next. Half of the companies on the S&P 500 are expected to be re-placed within the next 10 years, (Scott D. Anthony, S. Pat-rick Viguerie and Andrew Waldeck, “Corporate Longevity: Turbulence Ahead for Large Organizations,” Innosight). Fortune published its first-ever list of the top 500 companies in the country in 1955 (see a list on page 48). Since then, most of the list’s most storied names have found themselves absorbed by other corporations or have dropped off the list entirely. Only 60 of the original companies remain today.

Companies must change to survive. Eastman Kodak, No. 43 on Fortune’s first list, has tumbled in the ranks as photo-chemical film grew irrelevant in a digital age. The company was officially booted from the list in 2013, tumbling to a ranking of 966 by 2015. Idiosyncratic models are better able to capture the day-to-day changes that are ignored when generalizing.

The AI advantage: Causal analysis

Generalized population models force assumptions about individual companies. Apple and Microsoft are both high-tech industry giants that sell hardware, software and cloud services, but they are fundamentally different. They have different outlooks that appeal to different consumer segments. Instead of treating them as the same, AI can embrace what makes companies like these unique by using an idiosyncratic model.

Idiosyncratic models dig in deeper to account for a larger set of observable behaviors. Think of it as detective work. You can try to solve a murder by reading the latest crime statistics, which will tell you the hot spots where felonies are most often committed. You can find the other com-mon elements of each crime: the time of day, the number of accomplices and even the clearance rate (how likely it is that the crime will be solved). These facts tell you what happened. Yet you still have no idea who the culprit is or why the crime was committed.

In the classic TV drama formula, the alternative is to seek the motive, means and opportunity. Round up everyone who was in the neighborhood at the time of the crime (opportunity), then investigate each one and deter-mine whether they had both an incentive to commit the crime (motive) and the ability to do so (means). That is to say, you need to conduct an idiosyncratic analysis to zero in on the answer that tells you why something happened. This is fundamentally different from what you get from the nomothetic and statistical approach.

By creating an artificial population of several unique entities, the idiosyncratic model can simulate hypothetical causal relationships among the entities. This can be used to identify the causes of observed events and produce a theory that can be tested and verified.

For instance, Colonel Mustard was in the billiard room with a revolver when the crime was committed and he hated the victim. You can lay out the motive, means and opportunity. At the same time, the system has confirmed the other characters’ alibis. Colonel Mustard becomes the prime suspect through a process of elimination.

Unlike the sort of analysis provided by artificial intelligence based on mysterious deep learning models, knowledge-based idiosyncratic models can explain their predictions. There is no guesswork involved. The idiosyncratic model builds a case so that an action can be taken and the subsequent results evaluated. It creates a paper trail based on causality – a trail that can be followed and, more importantly, verified.

Idiosyncratic models are likely to be the next frontier of financial analysis. It would be foolish not to develop the competitive edge possible from capturing the “why” of individual company performance, or individual commodity values or any other financial instrument. AI offers better information that can increase the accuracy of analysis as well as the efficacy of financial decisions. Such a system will cut through the chaos with a fact-based assessment of likely future performance based on causal factors for each individual company, commodity, property or other financial instrument. Winning in a market where such systems exist would depend more on the skill in the application of this insight than on luck.

Considering the massive advantage that such a system offers, it would only be a matter of time before having this level of insight is a prerequisite for anyone hoping to succeed in the financial world.