Econometrics has been the main toolbox of economists for testing the empirical predictions of theoretical models. However, a topic that has taken into account much attention is the capability to use econometrics models to make future predictions. For example, it is a very well-known fact that the central bank has its inflation targeting regime based on the forecasted inflation rate in the Peruvian economy. Therefore, for an economist, making future predictions plays an essential role in policy subjects. In this line of view, Machine Learning (ML) models come into play since their main objective is to predict values of a variable y given a set of exogenous variables represented by X. This raises the following question: Why would economists keep using econometrics and not Machine Learning models?
To address this question, we must go back to our first econometrics course. When using econometrics, we are not only looking for forecasted values -even though they may be necessary for policy objectives-. We are also interested in other questions such as i) are our findings only a spurious relationship, or is there a causal effect between y and X? ii) test the empirical relationships that can be found in semi-structural and structural models, and iii) quantify marginal effects.
Some Machine Learning models have tried to address one or more of these questions, but a high cost: lack of interpretability, model complexity, and the risk of overfitting a model. To illustrate my point and give an example of overfitting, I will use an example given by Susan (2020). First, let’s imagine a model that wants to predict whether an image is a cat or a dog. For this purpose, the ML model will need images of cats and dogs, each with labels that classify them into two groups. Next, suppose that the most common pictures that can be found on the internet about cats are about them playing the piano. Therefore, if an object that looks like a piano is found in an image, the model will be most likely to predict a cat. As a result one may think that the model works in the sample, but what if dogs playing the piano becomes a trend? Then, the model will misclassify the pictures when using the model out of sample. The reason for this is intuitive: ML models use all available data and do not distinguish whether the picture contains a dog by itself or there is an object around. This can be a problem considering that most ML packages are black boxes, and when we use them, it is “either all in or nothing.” One may think that having a loss function that punishes if the results are guided by black stripes’ presence (absence) can be the solution. However, the programing skills required to implement this possible solution in the model are beyond most undergraduate and graduate-level programs.
This leads to the following consequence: most ML models require the ability to estimate the model faster than what patterns and trends change. To make this even more evident, let’s imagine a financial institution that wants to classify costumers in risk clusters. To accomplish this job, they use a complex ML model; however, it is constrained to change once a month due to regulation. Thus, if a shock happens and affects their client’s behavior, a completely new model may be required; however, the financial institution will have to wait until the next month due to regulation. In contrast to ML models, if a similar situation happens under an econometric model scheme, there are still results that can be relevant to define whether a customer belongs to one risk cluster or another. Therefore, there is not extremely necessary to re-estimate a new model that considers new complex relationships that can be found when dealing with large amounts of data.
Once we have identified that most ML models come with a huge cost, a lack of interpretability, and sometimes the necessity of re-estimating the model periodically, we can identify why economists are not likely to use ML models. When using econometrics, it is often thought that the “God of economics” exists, and he is the one that tells the researcher what the model looks like and which variables to include. Furthermore, when using econometrics, economists are looking for a causal relation that holds for long periods, and those do not depend on the sample that is being analyzed (external validity).
Notes based on:
Susan A., (2020), “Big Data, Machine Learning, and Artificial Intelligence: Methods Lectures and Applications”
Gabriel
Leave a Reply