Crypto Trading with Flink
Once I read an article on the internet about automating crypto trading, a topic that is hot those days. I can’t find this article anymore, but I remember that the author developed and described a program, basically a bot that was watching crypto currencies exchange rates and sending notifications when it is the right moment to buy or to sell. I was wondering what will it take to go one step further and automate the whole trading process.
This article is not about making a lot of money by trading cryptocurrency, although it could be a nice bonus. Rather, it is a summary of some technical challenges faced and the lessons learned while attempting to automate the trading process. My first goal was to fully automate the process, and as I had some experience with Flink, I decided to investigate whether it was suitable for this purpose. Trading involves monitoring price movements, making it a good example of a real-life event stream that can be handled by Flink, a tool used to process streaming data.
Trading platform
Based on our goals, the first step was to chose a cryptocurrency trading platform. Here is a short list of identified requirements for the exchange platform:
- API streaming cryptocurrency price movements
- API for buying and selling crypto assets
- Java client is nice to have but is optional
There are other non-technical requirements to consider when choosing a trading platform, such as registration requirements, trading fees, regulatory restrictions, and the security of your data and funds. Another factor worth considering is cryptocurrency offerings and trading amounts. From a technical standpoint, this means how much real-time data needs to be processed and how often it is generated.
I didn’t spend too much time searching or comparing different platforms. I just checked that Binance met all of the requirements, so I registered myself on it. After registering and making a small deposit, the last thing to do before using the platform was to generate a secret key, which is described in detail in the FAQ section. You can also restrict actions that can be performed using the secret key, and even restrict the usage of the secret key to your IP address only.
Another reason to choose Binance is that a Binance API Java client is available on GitHub. Moreover, the client supports user data and trade event streaming using web sockets. Listening to trade events is essentially the core functionality of automating trading, but receiving updates to user data, such as balance changes after order execution, is just as important.
Just note that you might need to clone the repository and compile the client on your own, as it might not be available as a Maven dependency from public repositories.
Later, I noticed that the developed solution is not limited to cryptocurrency trading platforms. Theoretically, it is possible to automate trading on any platform that provides the required set of APIs using the same approach. It is just that trends change often on cryptocurrency trading platforms, so depending on the trading strategy, events to enter or exit positions are happening frequently, and it is a nice technical challenge to catch them in time.
Flink Job
Typical Flink job is composed of sources responsible for reading data from an external data system, transformations performed on the data stream, such as filtering, aggregating, or joining data, and a sinks responsible for writing the output of the job to an external data system. Lets take a look which components must be implemented to automate crypto trading.
Source Functions
As previously mentioned, there are two or even three sources of events that need to be monitored for automated trading:
- trade events or price movements, which form the basis for trading, including trading volumes.
- user data needs to be monitored, which includes changes in the amounts of fiat or crypto assets after an order has been executed.
- additionally, user data event is a result of user account update, for example when user do an additional deposit on the platform.
Both types of events can be read from the web socket provided by the Binance API. However, Flink does not provide a source function to read this data directly as it does for example for reading from Kafka, so a custom source function must be implemented.
One of the initial requirement in this project was to monitor few crypto currencies at the same time. In terms of Flink this resulted in keyed streams. Each price bar is associated with a crypto asset. Each order close is also associated with a crypto asset. But fiat deposit event impacts and has to be broadcasted to all the keyed streams.
Just as a side note, the change in fiat and crypto amount can also be triggered by manually placing and executing a trade order. In such case the application must be able to reflect the change and take it into account on the following trading strategy calculation.
Process Functions
On each event the trading strategy is recalculated to answer the question whether to enter, to exit a position or to wait. The trading strategy requires at least a series of latest price bars, so the processing function needs to store the series by adding new price bars and removing outdated one. The length of the series depends on trading strategy.
Except the price bar series, the trading strategy calculation also require the current amount of fiat and crypto assets including the one in open positions, at least to check if there is enough money to enter and how much crypto must be sell to exit. This means that three types of source events must be merged together: price bar and order update event are both associated with a crypto asset and can be merged in a keyed stream, account update must be broadcasted to all keyed streams.
Sink Functions
Trading strategy recalculation based on the latest series of price bars and the amount of assets results in a decision to buy or sell. Initially, it might seem like sending this decision to the Binance API can be implemented as a sink function. However, such a function should not only make a REST API call but also handle the result of this call and update the list of open positions and the amount of available fiat and crypto assets.
The Binance API also supports order execution in testing mode, which is very useful while testing your trading strategy. Instead of placing and executing the order, the platform will only accept the order and check whether it is correctly filled or return an error if it is not correct. For example, there is not enough money in your account to buy crypto, or the amount of crypto is indicated with the wrong precision. The amount and price precision is another area to pay attention to while developing your trading bot.
Final notes
Trading automation or developing a trading bot is a challenging and complex technical project, made even more interesting by the high volatility of cryptocurrency prices, which results in frequently changing trading decisions.
Watching a dozen of crypto asset price movements generates a stream of events that can be handled using Apache Flink. At the same time it must be admitted that even if you are watching a dozens of crypto assets at the same time with price bars updated every second, the stream of events is not enormous. Well developed multi-tread Java application can handle this stream of events with ease.
Developing a trading bot as an Apache Flink application is also complex, as it requires merging three sources of events and internally storing and reflecting the list of currently opened positions on the trading platform. However, the benefits of a trading bot are numerous, including the ability to monitor the market 24/7 and catch opportunities that may arise while you’re sleeping.
It’s important to note that the success of automated trading depends on the selected trading strategy and is beyond the scope of this article. The goal here was to describe the technical challenges that must be overcome to develop a trading bot based on crypto assets.
