Machine Learning for Financial Risk Management with Python

Chapter 1. Fundamentals of Risk Management

In 2007, no one would have thought that risk functions could have changed as much as they have in the last eight years. It is a natural temptation to expect that the next decade has to contain less change. However, we believe that the opposite will likely be true.

Harle, Havas, and Samandari (2016)

Risk management is a constantly evolving process. Constant evolution is inevitable because long-standing risk management practice cannot keep pace with recent developments or be a precursor to unfolding crises. Therefore, it is important to monitor and adopt the changes brought by structural breaks in a risk management process. Adopting these changes implies redefining the components and tools of risk management, and that is what this book is all about.

Traditionally, empirical research in finance has had a strong focus on statistical inference. Econometrics has been built on the rationale of statistical inference. These types of models concentrate on the structure of underlying data, generating process and relationships among variables. Machine learning (ML) models, however, are not assumed to define the underlying data-generating processes but are considered as a means to an end for the purpose of prediction (Lommers, El Harzli, and Kim 2021). Thus, ML models tend to be more data centric and prediction accuracy oriented.

Moreover, data scarcity and unavailability have always been an issue in finance, and it is not hard to guess that the econometric models cannot perform well in those cases. Given the solution that ML models provide to data unavailability via synthetic data generation, these models have been on the top of the agenda in finance, and financial risk management is, of course, no exception.

Before going into a detailed discussion of these tools, it is worth introducing the main concepts of risk management, which I will refer to throughout the book. These concepts include risk, types of risks, risk management, returns, and some concepts related to risk management.

Risk

Risk is always out there, but understanding and assessing it is a bit tougher than knowing this due to its abstract nature. Risk is perceived as something hazardous, and it might be either expected or unexpected. Expected risk is something that is priced, but unexpected risk can be barely accounted for, so it might be devastating.

As you can imagine, there is no general consensus on the definition of risk. However, from the financial standpoint, risk refers to a likely potential loss or the level of uncertainty to which a company can be exposed. McNeil, Alexander, and Paul (2015) define risk differently, as:

Any event or action that may adversely affect an organization’s ability to achieve its objectives and execute its strategies or, alternatively, the quantifiable likelihood of loss or less-than-expected returns.

These definitions focus on the downside of the risk, implying that cost goes hand in hand with risk, but it should also be noted that there is not necessarily a one-to-one relationship between them. For instance, if a risk is expected, a cost incurred is relatively lower (or even ignorable) than that of unexpected risk.

Return

All financial investments are undertaken to gain profit, which is also called return. More formally, return is the gain made on an investment in a given period of time. Thus, return refers to the upside of the risk. Throughout the book, risk and return will refer to downside and upside risk, respectively.

As you can imagine, there is a trade-off between risk and return: the higher the assumed risk, the greater the realized return. As it is a formidable task to come up with an optimum solution, this trade-off is one of the most controversial issues in finance. However, Markowitz (1952) proposes an intuitive and appealing solution to this long-standing issue. The way he defines risk, which was until then ambiguous, is nice and clean and led to a shift in landscape in financial research. Markowitz used standard deviation ${\sigma }_{R_i}$ to quantify risk. This intuitive definition allows researchers to use mathematics and statistics in finance. The standard deviation can be mathematically defined as (Hull 2012):

$$\sigma =\sqrt{𝔼\left(R^2\right)-{\left[𝔼\left(R\right)\right]}^2}$$

where R and $𝔼$ refer to annual return and expectation, respectively. This book uses the symbol $𝔼$ numerous times as expected return represents the return of interest. This is because it is probability we are talking about in defining risk. When it comes to portfolio variance, covariance comes into the picture, and the formula turns out to be:

$${\sigma }_p^2=w_a^2{\sigma }_a^2+w_b^2{\sigma }_b^2+2w_a{w}_b\text{Cov}(r_a,r_b)$$

where w denotes weight, ${\sigma }^2$ is variance, and Cov is covariance matrix.

Taking the square root of the variance obtained previously gives us the portfolio standard deviation:

$${\sigma }_p=\sqrt{{\sigma }_p^2}$$

In other words, portfolio expected return is a weighted average of the individual returns and can be shown as:

$𝔼\left(R\right)={\sum }_i^n{w}_i{R}_i=w_1{R}_1+w_2{R}_2\cdots +w_n{R}_n$

Let us explore the risk-return relationship by visualization. To do that, a hypothetical portfolio is constructed to calculate necessary statistics with Python:

In [1]: import statsmodels.api as sm
        import numpy as np
        import plotly.graph_objs as go
        import matplotlib.pyplot as plt
        import plotly
        import warnings
        warnings.filterwarnings('ignore')

In [2]: n_assets = 5 
        n_simulation = 500 

In [3]: returns = np.random.randn(n_assets, n_simulation) 

In [4]: rand = np.random.rand(n_assets) 
        weights = rand/sum(rand) 

        def port_return(returns):
            rets = np.mean(returns, axis=1)
            cov = np.cov(rets.T, aweights=weights, ddof=1)
            portfolio_returns = np.dot(weights, rets.T)
            portfolio_std_dev = np.sqrt(np.dot(weights, np.dot(cov, weights)))
            return portfolio_returns, portfolio_std_dev 

In [5]: portfolio_returns, portfolio_std_dev = port_return(returns) 

In [6]: print(portfolio_returns)
        print(portfolio_std_dev) 

        0.012968706503879782
        0.023769932556585847

In [7]: portfolio = np.array([port_return(np.random.randn(n_assets, i))
                              for i in range(1, 101)]) 

In [8]: best_fit = sm.OLS(portfolio[:, 1], sm.add_constant(portfolio[:, 0]))\
                   .fit().fittedvalues 

In [9]: fig = go.Figure()
        fig.add_trace(go.Scatter(name='Risk-Return Relationship',
                                 x=portfolio[:, 0],
                                 y=portfolio[:, 1], mode='markers'))
        fig.add_trace(go.Scatter(name='Best Fit Line',
                                 x=portfolio[:, 0],
                                 y=best_fit, mode='lines'))
        fig.update_layout(xaxis_title = 'Return',
                          yaxis_title = 'Standard Deviation',
                          width=900, height=470)
        fig.show() 

Number of assets considered

Number of simulations conducted

Generating random samples from normal distribution used as returns

Generating random number to calculate weights

Calculating weights

Function used to calculate expected portfolio return and portfolio standard deviation

Calling the result of the function

Printing the result of the expected portfolio return and portfolio standard deviation

Rerunning the function 100 times

To draw the best fit line, run linear regression

Drawing interactive plot for visualization purposes

Figure 1-1, generated via the previous Python code, confirms that the risk and return go in tandem, but the magnitude of this correlation varies depending on the individual stock and the financial market conditions.

Figure 1-1. Risk-return relationship

Risk Management

Financial risk management is a process to deal with the uncertainties resulting from financial markets. It involves assessing the financial risks facing an organization and developing management strategies consistent with internal priorities and policies (Horcher 2011).

According to this definition, as every organization faces different types of risks, the way that a company deals with risk is completely unique. Every company should properly assess and take necessary action against risk. This does not necessarily mean, however, that once a risk is identified, it needs to be mitigated as much as a company can.

Risk management is, therefore, not about mitigating risk at all costs. Mitigating risk may require sacrificing return, and it can be tolerable up to certain level as companies search for higher return as much as lower risk. Thus, to maximize profit while lowering the risk should be a delicate and well-defined task.

Managing risk comes with a cost, and even though dealing with it requires specific company policies, there exists a general framework for possible risk strategies:

Ignore

In this strategy, companies accept all risks and their consequences and prefer to do nothing.

Transfer

This strategy involves transferring the risks to a third party by hedging or some other way.

Mitigate

Companies develop a strategy to mitigate risk partly because its harmful effect might be considered too much to bear and/or surpass the benefit attached to it.

Accept

If companies embrace the strategy of accepting the risk, they properly identify risks and acknowledge the benefit of them. In other words, when assuming certain risks arising from some activities bring value to shareholders, this strategy can be chosen.

Main Financial Risks

Financial companies face various risks over their business life. These risks can be divided into different categories in a way to more easily identify and assess them. These main financial risk types are market risk, credit risk, liquidity risk, and operational risk, but again, this is not an exhaustive list. However, we confine our attention to the main financial risk types throughout the book. Let’s take a look at these risk categories.

Market risk

This risk arises due to a change in factors in the financial market. For instance, an increase in interest rate might badly affect a company that has a short position.

A second example can be given about another source of market risk: exchange rate. A company involved in international trade, whose commodities are priced in US dollars, is highly exposed to a change in US dollars.

As you can imagine, any change in commodity price might pose a threat to a company’s financial sustainability. There are many fundamentals that have a direct effect on commodity price, including market players, transportation cost, and so on.

Credit risk

Credit risk is one of the most pervasive risks. It emerges when a counterparty fails to honor debt. For instance, if a borrower is unable to make a payment, then credit risk is realized. Deterioration of credit quality is also a source of risk through the reduced market value of securities that an organization might own (Horcher 2011).

Liquidity risk

Liquidity risk had been overlooked until the 2007–2008 financial crisis, which hit the financial market hard. From that point on, research on liquidity risk has intensified. Liquidity refers to the speed and ease with which an investor executes a transaction. This is also known as trading liquidity risk. The other dimension of liquidity risk is funding liquidity risk, which can be defined as the ability to raise cash or availability of credit to finance a company’s operations.

If a company cannot turn its assets into cash within a short period of time, this falls under the liquidity risk category, and it is quite detrimental to the company’s financial management and reputation.

Operational risk

Managing operational risk is not a clear and foreseeable task, and it takes up a great deal of a company’s resources due to the intricate and internal nature of the risk. Questions include:

• How do financial companies do a good job of managing risk?

• Do they allocate necessary resources for this task?

• Is the importance of risk to a company’s sustainability gauged properly?

As the name suggests, operational risk arises when external events or inherent operation(s) in a company or industry pose a threat to the day-to-day operations, profitability, or sustainability of that company. Operational risk includes fraudulent activities, failure to adhere to regulations or internal procedures, losses due to lack of training, and so forth.

Well, what happens if a company is exposed to one or more than one of these risks and is unprepared? Although it doesn’t happen frequently, historical events tell us the answer: the company might default and run into a big financial collapse.

Big Financial Collapse

How important is risk management? This question can be addressed by a book with hundreds of pages, but in fact, the rise of risk management in financial institutions speaks for itself. For example, the global financial crisis of 2007–2008 has been characterized as a “colossal failure of risk management” (Buchholtz and Wiggins 2019), though this was really just the tip of the iceberg. Numerous failures in risk management paved the way for this breakdown in the financial system. To understand this breakdown, we need to dig into past financial risk management failures. A hedge fund called Long-Term Capital Management (LTCM) presents a vivid example of a financial collapse.

LTCM formed a team with top-notch academics and practitioners. This led to a fund inflow to the firm, and it began trading with $1 billion. By 1998, LTCM controlled over $100 billion and was heavily invested in some emerging markets, including Russia. The Russian debt default deeply affected LTCM’s portfolio due to flight to quality,¹ and it took a severe blow, which led it to bust (Bloomfield 2003).

Metallgesellschaft (MG) is another company that no longer exists due to bad financial risk management. MG largely operated in gas and oil markets. Because of its high exposure, MG needed funds in the aftermath of the large drop in gas and oil prices. Closing the short position resulted in losses around $1.5 billion.

Amaranth Advisors (AA) is another hedge fund that went into bankruptcy due to heavily investing in a single market and misjudging the risks arising from these investments. By 2006, AA had attracted roughly $9 billion of assets under management but lost nearly half of it because of the downward move in natural gas futures and options. The default of AA is attributed to low natural gas prices and misleading risk models (Chincarini 2008).

Stulz’s paper, “Risk Management Failures: What Are They and When Do They Happen?” (2008) summarizes the main risk management failures that can result in default:

• Mismeasurement of known risks

• Failure to take risks into account

• Failure in communicating risks to top management

• Failure in monitoring risks

• Failure in managing risks

• Failure to use appropriate risk metrics

Thus, the global financial crisis was not the sole event that led regulators and institutions to redesign their financial risk management. Rather, it is the drop that filled the glass, and in the aftermath of the crisis, both regulators and institutions have adopted lessons learned and improved their processes. Eventually, this series of events led to a rise in financial risk management.

Information Asymmetry in Financial Risk Management

Although it is theoretically intuitive, the assumption of a completely rational decision maker, the main building block of modern finance theory, is too perfect to be real. Behavioral economists have therefore attacked this idea, asserting that psychology plays a key role in the decision-making process:

Making decisions is like speaking prose—people do it all the time, knowingly or unknowingly. It is hardly surprising, then, that the topic of decision making is shared by many disciplines, from mathematics and statistics, through economics and political science, to sociology and psychology.

Kahneman and Tversky (1984)

Information asymmetry and financial risk management go hand in hand as the cost of financing and firm valuation are deeply affected by information asymmetry. That is, uncertainty in valuation of a firm’s assets might raise the borrowing cost, posing a threat to a firm’s sustainability (see DeMarzo and Duffie 1995 and Froot, Scharfstein, and Stein 1993).

Thus, the roots of the failures described previously lie deeper in such a way that a perfect hypothetical world in which a rational decision maker lives is unable to explain them. At this point, human instincts and an imperfect world come into play, and a mixture of disciplines provides more plausible justifications. Adverse selection and moral hazard are two prominent categories accounting for market failures.

Adverse Selection

Adverse selection is a type of asymmetric information in which one party tries to exploit its informational advantage. This arises when sellers are better informed than buyers. This phenomenon was perfectly coined by Akerlof (1978) as “the Markets for Lemons.” Within this framework, “lemons” refer to low-quality commodities.

Consider a market with lemons and high-quality cars, and buyers know that they’re likely to buy a lemon, which lowers the equilibrium price. However, the seller is better informed whether the car is a lemon or of high quality. So, in this situation, benefit from exchange might disappear, and no transaction takes place.

Because of its complexity and opaqueness, the mortgage market in the pre-crisis era is a good example of adverse selection. Borrowers knew more about their willingness and ability to pay than lenders. Financial risk was created through the securitizations of the loans (i.e., mortgage-backed securities). From that point on, the originators of the mortgage loans knew more about the risks than those who were selling them to investors in the form of mortgage-backed securities.

Let’s try to model adverse selection using Python. It is readily observable in the insurance industry, and therefore I would like to focus on that industry to model adverse selection.

Suppose that the consumer utility function is:

$$U\left(x\right)=e^{\gamma x}$$

where x is income and $\gamma$ is a parameter, which takes on values between 0 and 1.

Note

The utility function is a tool used to represent consumer preferences for goods and services, and it is concave for risk-averse individuals.

The ultimate aim of this example is to decide whether or not to buy an insurance based on consumer utility.

For the sake of practice, I assume that the income is US$2 and the cost of the accident is US$1.5.

Now it is time to calculate the probability of loss, $\pi$, which is exogenously given and uniformly distributed.

As a last step, to find equilibrium, I have to define supply and demand for insurance coverage. The following code block indicates how we can model the adverse selection:

In [10]: import matplotlib.pyplot as plt
         import numpy as np
         plt.style.use('seaborn')

In [11]: def utility(x):
             return(np.exp(x ** gamma)) 

In [12]: pi = np.random.uniform(0,1,20)
         pi = np.sort(pi) 

In [13]: print('The highest three probability of losses are {}'
               .format(pi[-3:])) 
         The highest three probability of losses are [0.834261   0.93542452
          0.97721866]

In [14]: y = 2
         c = 1.5
         Q = 5
         D = 0.01
         gamma = 0.4

In [15]: def supply(Q):
             return(np.mean(pi[-Q:]) * c) 

In [16]: def demand(D):
             return(np.sum(utility(y - D) > pi * utility(y - c) + (1 - pi)
                           * utility(y))) 

In [17]: plt.figure()
         plt.plot([demand(i) for i in np.arange(0, 1.9, 0.02)],
                  np.arange(0, 1.9, 0.02),
                  'r', label='insurance demand')
         plt.plot(range(1,21), [supply(j) for j in range(1,21)],
                  'g', label='insurance supply')
         plt.ylabel("Average Cost")
         plt.xlabel("Number of People")
         plt.legend()
         plt.show()

Writing a function for risk-averse utility function

Generating random samples from uniform distribution

Picking the last three items

Writing a function for supply of insurance contracts

Writing a function for demand of insurance contracts

Figure 1-2 shows the insurance supply-and-demand curve. Surprisingly, both curves are downward sloping, implying that as more people demand contracts and more people are added on the contracts, the risk lowers, affecting the price of the contract.

The straight line presents the insurance supply and average cost of the contracts and the other line, showing a step-wise downward slope, denotes the demand for insurance contracts. As we start analysis with the risky customers, as you add more and more people to the contract, the level of riskiness diminishes in parallel with the average cost.

Figure 1-2. Adverse selection

Moral Hazard

Market failures also result from asymmetric information. In a moral hazard situation, one party of the contract assumes more risk than the other party. Formally, moral hazard may be defined as a situation in which the more informed party takes advantages of the private information at their disposal to the detriment of others.

For a better understanding of moral hazard, a simple example can be given from the credit market: suppose that entity A demands credit for use in financing the project that is considered feasible to finance. Moral hazard arises if entity A uses the loan for the payment of credit debt to bank C, without prior notice to the lender bank. While allocating credit, the moral hazard situation that banks may encounter arises as a result of asymmetric information, decreases banks’ lending appetites, and appears as one of the reasons why banks put so much labor into the credit allocation process.

Some argue that rescue operations undertaken by the Federal Reserve Board (Fed) for LTCM can be considered a moral hazard in the way that the Fed enters into contracts in bad faith.

Conclusion

This chapter presented the main concepts of financial risk management with a view to making sure that we are all on the same page. These terms and concepts will be used frequently throughout this book.

In addition, a behavioral approach, attacking the rationale of a finance agent, was discussed so that we have more encompassing tools to account for the sources of financial risk.

In the next chapter, we will discuss the time-series approach, which is one of the main pillars of financial analysis in the sense that most financial data has a time dimension, which requires special attention and techniques to deal with.

References

Articles and chapters cited in this chapter:

• Akerlof, George A. 1978. “The Market for Lemons: Quality Uncertainty and the Market Mechanism.” Uncertainty in Economics, 235-251. Academic Press.

• Buchholtz, Alec, and Rosalind Z. Wiggins. 2019. “Lessons Learned: Thomas C. Baxter, Jr., Esq.” Journal of Financial Crises 1, no. (1): 202-204.

• Chincarini, Ludwig. 2008. “A Case Study on Risk Management: Lessons from the Collapse of Amaranth Advisors Llc.” Journal of Applied Finance 18 (1): 152-74.

• DeMarzo, Peter M., and Darrell Duffie. 1995. “Corporate Incentives for Hedging and Hedge Accounting.” The Review of Financial Studies 8 (3): 743-771.

• Froot, Kenneth A., David S. Scharfstein, and Jeremy C. Stein. 1993. “Risk Management: Coordinating Corporate Investment and Financing Policies.” The Journal of Finance 48 (5): 1629-1658.

• Harle, P., A. Havas, and H. Samandari. 2016. The Future of Bank Risk Management. McKinsey Global Institute.

• Kahneman, D., and A. Tversky. 1984. “Choices, Values, and Frames. American Psychological Association.” American Psychologist, 39 (4): 341-350.

• Lommers, Kristof, Ouns El Harzli, and Jack Kim. 2021. “Confronting Machine Learning With Financial Research.” Available at SSRN 3788349.

• Markowitz H. 1952. “Portfolio Selection”. The Journal of Finance. 7 (1): 177—91.

• Stulz, René M. 2008. “Risk Management Failures: What Are They and When Do They Happen?” Journal of Applied Corporate Finance 20 (4): 39-48.

Books cited in this chapter:

• Bloomfield, S. 2013. Theory and Practice of Corporate Governance: An Integrated Approach. Cambridge: Cambridge University Press.

• Horcher, Karen A. 2011. Essentials of Financial Risk Management. Vol. 32. Hoboken, NJ: John Wiley and Sons.

• Hull, John. 2012. Risk Management and Financial Institutions. Vol. 733. Hoboken, NJ: John Wiley and Sons.

• McNeil, Alexander J., Rüdiger Frey, and Paul Embrechts. 2015. Quantitative Risk Management: Concepts, Techniques and Tools, Revised edition. Princeton, NJ: Princeton University Press.

¹ Flight to quality refers to a herd behavior in which investors stay away from risky assets such as stocks and take long positions in safer assets such as government-issued bonds.