We are excited to announce the release of MLFinlab v2.2.0!

Coinciding with our recent closed beta announcement of integrating MLFinlab with Docker (please go sign up if you’re interested!), we’re excited to announce MLFinlab v2.2.0 that adds support for Python 3.10 and 3.11. This now brings support to every major currently-supported version of Python.

We hope this will make using MLFinlab in your various production environments easier in the future.

As always, for installation instructions, please see the official documentation (which is available from the Hudson & Thames portal if you have an active subscription).

We’re excited to announce the latest release of MLFinlab, our suite of composable components and production-ready algorithms for developing trading strategies that leverage machine learning.

Highlights of this release includes Python 3.9 support, usability improvements to our volatility estimators, productivity improvements when generating bars from tick data, as well as a brand new API reference page. Some smaller changes include updates to more examples, and converting additional examples to doctest for increased reliability and testability.

We’ll break down some of the major changes below, and how they will impact users who are upgrading.

Python 3.9 support

This release officially marks the inclusion of Python 3.9 support to MLFinlab. This has been a sought-after improvement for some time, and we are happy to finally announce its general availability. This also lays the foundation for future Python 3.10 support, which is currently on the roadmap.

DatetimeIndex for creating bars

We’ve updated the behaviour of the data_structures module to now set the date_time column as a DatetimeIndex, instead of a separate column:

>>> from mlfinlab.data_structures import time_data_structures
>>> # Get processed tick data csv from url
>>> tick_data_url = "https://raw.githubusercontent.com/hudson-and-thames/example-data/main/processed_tick_data.csv"
>>> time_bars = time_data_structures.get_time_bars(
...     tick_data_url, resolution="D", verbose=False
... )
>>> time_bars  
            tick_num    open   high    low   close  volume  cum_buy_volume  cum_ticks  cum_dollar_value
date_time
2023-03-02     13642  804.25  808.0  803.0  807.00   49208           25816      13641      3.963169e+07
2023-03-03     66322  807.00  812.5  806.0  810.75  171378           94591      52680      1.388561e+08

Why have we made this change? Most of the downstream functions expect to receive input as a DataFrame, where the index is the date_time. Users would have to manually set the date_time column as the index each and every time after creating their bars. This is tedious, is easily forgotten, and can lead to cryptic downstream errors. As a result, we’ve updated the default behaviour to do this automatically.

We’ve also updated all of our examples to reflect this new behaviour.

If your current code does manually set the date_time as the index, you’ll need to remove these parts from your code to prevent an error from being raised. Fortunately removing the code shouldn’t be a difficult process.

Improvements to volatility estimators

Previously, volatility estimators in our relied on the implicit column order when performing calculations. This is error-prone, and wasn’t documented clearly. Unfortunately this made it possible to get erroneous results without warning. As a result, we’ve updated the volatility estimators to instead use column names directly, thus ensuring correct calculations. Depending on the estimator in question, we now check for the following lowercase column names in order to perform the estimation: 'open', 'high', 'low', 'close'. This may require you to update your code, depending on how you’ve named your columns.

We’ve also updated all of our examples to reflect this new approach, so you can refer to them for additional context:

>>> import pandas as pd
>>> import yfinance as yf
>>> from mlfinlab.features.volatility_estimators import parkinson
>>> # Retrieve the DataFrame with time series of returns
>>> ohlc = pd.read_csv(
...     "https://raw.githubusercontent.com/hudson-and-thames/example-data/main/yahoo_finance_SPY_2012-03-26_to_2023-12-06.csv",
...     parse_dates=[0],
...     index_col=0,
... )
>>> ohlc.columns = [
...     col.lower() for col in ohlc.columns
... ]  # volatility estimators expect lower-case column names
>>> list(ohlc.columns)
['open', 'high', 'low', 'close', 'adj close', 'volume']
>>> # Calculate volatility with Parkinson estimator (Sinclair formula)
>>> parkinson_sinclair = parkinson(ohlc, 22, False)
>>> # Calculate volatility with Parkinson estimator (De Prado formula)
>>> parkinson_deprado = parkinson(ohlc, 22, True)

API Reference Docs

We’ve added an API reference to our documentation, which serves as a useful reference for every single module, class and function in MLFinlab. If you’re already familiar with a component, and just need to reference the API, this is a quick way of doing so without having to scroll through the entire user-guide. It’s also a new way to explore what’s on offer in MLFinlab. A demo of the API reference can be seen below:

We believe this is a very useful addition to the documentation.

Changes to ml_cross_val_score

We’ve slightly tweaked the ml_cross_val_score function to now require the require_proba argument as input. Previously, require_proba had a default value of False, which led to accidental errors and user confusion when passing in a scoring function that required probabilities as input rather than labels. Since the argument was optional, users were unaware that they had to do a explicitly specify require_proba=True. This has now been rectified, and require_proba is no longer an optional argument.

Other improvements

We’ve updated a number of examples, and updated links to our notebooks, which are now available on a public repository. Not everything has been migrated across yet, given the immense size of our documentation, but we will continue to steadily do so as we continue to improve and update the various modules in MLFinlab.

Upgrading

As always, you can follow the installation instructions in the documentation for installing the latest version of MLFinlab.

Hudson & Thames has consistently been at the forefront of delivering innovation in the quantitative finance landscape. Our flagship offering, MLFinLab, is a testament to our commitment, presenting a robust Python library teeming with production-ready algorithms derived from top-tier academic journals and advanced textbooks. It’s an essential tool, uniquely designed to empower portfolio managers and traders by providing reproducible, interpretable, and easy-to-use tools.

Today, we’re attempting to make your MLFinLab experience even more seamless. We’re thrilled to announce the upcoming closed beta launch for the Dockerized version of MLFinLab.

Tackling Python’s Challenges with Docker

Python, as powerful and flexible as it is, does present some unique challenges, one of which is the infamous “Python dependency hell”. This phrase refers to the complications encountered when managing the different versions and dependencies that Python applications require. Navigating these issues can be a time-consuming and complex task, especially when dealing with large-scale, intricate projects like MLFinLab.

Moreover, in managed environments, you may find pre-installed packages that can potentially conflict with other dependencies, causing incompatibility issues that can impact the stability of your application. Even though cross-platform compatibility can be achieved, it often requires considerable manual effort and introduces more variables that could affect overall reliability.

This is why we’d like to introduce Docker.

Reimagining MLFinLab with Docker

Docker is designed to package and isolate applications with their dependencies into a standardized unit for software development. By distributing MLFinlab in a Docker container, we can effectively overcome the complexities of Python dependency management. Docker containers bundle the MLFinLab application and all its dependencies into one easily distributable and quick-to-set-up package. For our users, this means you’ll spend less time wrestling with system-specific issues and fighting dependencies and more time leveraging MLFinLab’s robust features.

The Dockerized version of MLFinLab brings the promise of cross-platform compatibility and reliability. You can run MLFinLab on any system that supports Docker, be it Linux, Windows, or macOS. No more cross-platform inconsistencies or version conflicts, just a smooth, reliable application experience. The incorporation of Docker also paves the way for exciting new deployment options in the future, offering an unparalleled level of flexibility (particularly if you’re an institutional quant, where your development environment is managed by an IT team — but we’ll have more to say about this soon).

What will this practically look like?

Forget about the cumbersome process of creating a new Python environment and painstakingly installing MLFinLab with all of its dependencies using pip. Instead, imagine an effortless system that does all the work for you. That’s precisely what we’re bringing to you with our Dockerized MLFinLab.

You’ll simply pull a Docker image from the cloud, already fully loaded with all the necessary dependencies. As you initiate the container, we’ll verify your API key and seamlessly pull and install MLFinLab inside the container. Following this, a Jupyter Lab or Notebook instance will be launched, all ready and set for you to use.

What’s truly remarkable about this process is its isolation from your actual system. Everything takes place within the Docker container, leaving your system pristine and untouched. This means no more concerns about managing dependencies or ensuring a locally working version of Python.

In essence, the Dockerized MLFinLab delivers a radically simplified installation experience, one that we believe holds significant advantages over the traditional installation process of MLFinLab. It’s a game-changer, designed with user convenience at its core.

Join the Closed Beta Testing Phase

We’re now inviting all our users to participate in this exciting development by joining our closed beta testing phase. While the opportunity to sign up is open to everyone, our existing customers will be given priority access as a token of our appreciation for their ongoing support. Feedback is a vital part of this process—your insights will be instrumental in helping us enhance and perfect this Dockerized version of MLFinLab.

We’re eager to hear your insights, feedback, and suggestions. The timeline for the beta testing phase extends over the next few months, followed by the official release if we reach the level of reliability and performance , which will be guided and informed by your invaluable feedback.

An Exciting Leap Forward

The Dockerized version of MLFinLab represents a significant leap forward in quantitative finance software. With the promise of simplified deployment, enhanced cross-platform compatibility, and faster setup, it’s set to become an invaluable tool for portfolio managers, traders, and quantitative analysts alike.

Sign up for our closed beta test today and play an integral part in shaping the future of MLFinLab. We can’t wait to hear your feedback.

To those who are interested, you can sign up to the beta using the button below, and we’ll be in touch when we’re ready to launch.

We are excited to announce the release of ArbitrageLab v0.8!

This is primarily a maintenance release, where we have updated the majority of our dependencies to their latest stable version. This ensures we maintain the maximum reliability and security possibly, while also enabling users to remain compatible with up-to-date packages in their development environment. This release also lays the foundation for future developments on ArbitrageLab, which we’ll be announcing in the future.

We recommended creating a new virtual environment when upgrading, given the large upgrade in dependencies.

For instructions on installing ArbitrageLab, you can find the link to the documentation via the Portal.