How the TIC algorithm works?

In this section, we will be going through a quick summary of the steps involved in computing the TIC matrix.

Fitting our Tree Graph Structure

In this algorithm, the tree graph represents the economic theory and hierarchical structure of our assets. This tree structure is fit based upon our empirical correlation matrix. Essentially, this structure will tell us which assets are closely related to each other and the relative distance between them. This results in a binary tree using an agglomerative clustering technique which can be visualized through a dendrogram. However, this is not the final tree graph we are looking for. The dendrogram always has two items per cluster while our actual tree graph could have one or more leaves per branch. Additionally, the dendrogam will always have  clusters while the tree graph may have an unlimited number of levels and our dendrogram incorporates the notion of distance but the tree graph does not.

The general steps for this part of the algorithm are as follows:

1. If there is no top level of the tree (tree root), this level is added so that all variables are included in one general cluster.
2. The empirical correlation matrix is transformed into a matrix of distances using the above formula:\begin{aligned}d(i, j) = \sqrt{0.5 * (1 – \rho(i,j))}\end{aligned}
3. For each level of the tree, the elements are grouped by elements from the higher level. The algorithm iterates from the lowest to the highest level of the tree.
4. A linkage object is created for these grouped elements based on their distance matrix. Each link in the linkage object is an array representing a cluster of two elements and has the following data as elements.
   • ID of the first element in a cluster
   • ID of the second element in a cluster
   • Distance between the elements
   • Number of atoms (simple elements from the portfolio and not clusters) inside
5. A linkage object is transformed to reflect the previously created clusters.
6. A transformed local linkage object is added to the global linkage object.
7. Distance matrix is adjusted to the newly created clusters – elements that are now in the new clusters are replaced by the clusters in the distance matrix. The distance from the new clusters to the rest of the elements in the distance matrix is calculated as a weighted average of distances of two elements in a cluster to the other elements. The weight is the number of atoms in an element. So, the formula is:\begin{aligned}DistanceCluster = \frac{Distance_{1}*NumAtoms_{1} + Distance_{2}*NumAtoms_{2}}{NumAtoms_{1} + NumAtoms_{2}}\end{aligned}

The linkage object, representing a dendrogram of all elements in a portfolio is the result of the first step of the algorithm. It sequentially clusters two elements together, while measuring how closely together the two elements are, until all elements are subsumed within the same cluster. The final object looks something like this:

Deriving the Correlation Matrix

We can now derive a correlation matrix from our linkage object returned by the first step.

1. One by one, the clusters (each represented by a link in the linkage object) are decomposed to lists of atoms contained in each of the two elements of the cluster.
2. The elements on the main diagonal of the resulting correlation matrix are set to 1s. The off-diagonal correlations between the variables are computed as:\begin{aligned}\rho(i, j) = 1 – 2*d^{2}(i, j)\end{aligned}

De-Noising the Correlation Matrix

In our last step, the correlation matrix is de-noised.

1. The eigenvalues and eigenvectors of the correlation matrix are calculated.
2. Marcenko-Pastur distribution is fit to the eigenvalues of the correlation matrix and the maximum theoretical eigenvalue is calculated.
3. This maximum theoretical eigenvalue is set as a threshold and all the eigenvalues above the threshold are shrinked.
4. The de-noised correlation matrix is calculated back from the eigenvectors and the new eigenvalues.

Using PortfolioLab’s TIC Implementation

In this section, we will go through a working example of using the TIC implementation provided by PortfolioLab and test it on a portfolio of assets.

import portfoliolab as pl
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

Specifying the Economic Structure a.k.a Tree Graph

The TIC algorithm combines both an economic theory and empirical data. The economic theory is represented by a tree graph and our empirical data is represented by an empirical correlation matrix.

As mentioned previously, the tree graph embeds important information about the hierarchy of the assets in our portfolio and this comes from external market knowledge. Let us look at how to create the graph file.

An example of a hierarchical structure for the financial instruments is the MSCI’s Global Industry Classification Standard (GICS) that classifies investment universes in terms of four nested levels – sub-industry, industry, industry group, and sector. Investors can also add more diverse levels in the tree such as business ties, supply chain, competitors, industrial sectors, shared ownership, etc… For this blog post, we are using a dataset of ETFs and have created an example dataframe representing the tree graph:

# Getting the tree graph for ETFs 
etf_tree = pd.read_csv("classification_tree.csv") 

# Having a loot at the tree structure of ETFs 
print(etf_tree)

The proposed tree structure has four levels:

1. Ticker of the ETF
2. Sector of the ETF
   • XXXX10 – General/No sector
   • XXXX20 – Materials
   • XXXX30 – Energy
   • XXXX40 – Financial
   • XXXX50 – Technology
   • XXXX60 – Utilities
3. Region of the ETF
   • XX10 – Emerging markets
   • XX20 – Europe
   • XX30 – US and Canada
   • XX40 – Asia
4. Type of the ETF
   • 10 – Equity ETF
   • 20 – Bond ETF

The columns are ordered bottom-up, with the leftmost column corresponding to the terminal leaves, and the rightmost column corresponding to the tree’s root. This means that if you converted this dataframe into a tree, then the topmost level would be the type of the asset (in this case, ETF) with the subsequent nested levels being region, sector and finally the asset itself.

Calculating the Empirical Correlation Matrix

Now we calculate the series of returns from the raw prices and finally get the empirical correlations.

# Getting the price data for ETFs 
etf_prices = pd.read_csv('stock_prices.csv', parse_dates=True, index_col='Date')

# Class with returns calculation function
ret_est = pl.estimators.ReturnsEstimators()

# Calculating returns
etf_returns = ret_est.calculate_returns(etf_prices)

# Now the emperical correlation matrix can be calculated
etf_corr = etf_returns.corr()

Calculating the TIC Matrix

Equipped with the tree graph and the empirical correlation matrix, we merge them using the TIC algorithm to generate the new correlation matrix.

# Calculating the relation of sample length T to the number of variables N
# It's used for de-noising the TIC matrix
tn_relation = etf_returns.shape[0] / etf_returns.shape[1]

# Class with TIC function
tic = pl.estimators.TheoryImpliedCorrelation()

# Calculating theory-implied correlation matrix
etf_tic = tic.tic_correlation(etf_tree, etf_corr, tn_relation)

# Setting the indexes of the theory-implied correlation matrix
etf_tic = pd.DataFrame(etf_tic, index=etf_corr.index, columns=etf_corr.index)

Visualising the TIC Matrix

We can visualize the difference between the Empirical correlation matrix and the Theory-implied correlation matrix using heatmaps.

# Plotting the heatmap of the matrix
sns.heatmap(etf_corr, cmap="Greens")
plt.title('Empirical Correlation Matrix')
plt.show()

sns.heatmap(etf_tic, cmap="Greens")
plt.title('Theory-Implied Correlation Matrix')
plt.show()