Algo Trading with REST API and Python | Part 4: Building and Backtesting an EMA Crossover Strategy

Using FXCM’s REST API and the fxcmpy Python wrapper makes it quick and easy to create actionable trading strategies in a matter of minutes.  In this article we will be building a strategy and backtesting that strategy using a simple backtester on historical data. You can download the completed Python backtest from our Github.

One important note to consider before jumping into the material is that backtested results are hypothetical and may not reflect the true performance of a system, as past performance is not indicative of future returns. Additionally, the backtester we will create today is meant to give us a quick understanding of a strategy’s behavior overall and not to simulate every aspect of a strategy’s execution (such as transaction costs, market impact, price slippage, etc). In future articles we will cover how to build a more robust backtesting system, so check back often for more!

Importing the Necessary Packages

To get started on building our strategy, we will begin by importing the necessary packages, each of which can quickly be installed using the “pip install” command from your command prompt if you don’t already have them. We will be using fxcmpy to pull historical prices, pandas and numpy for analyzing our time series data, pyti for quick access to technical indicators, and matplotlib for visualizing our results.

import fxcmpy 
import pandas as pd
import numpy as np
import datetime as dt
import matplotlib.pyplot as plt
%matplotlib inline
from pyti.exponential_moving_average import exponential_moving_average as ema

Once we have each of the above packages imported, it’s time to move on to pulling historical data using our FXCM Real or Demo account.

Connecting and Retrieving Historical Prices

In order to connect to the FXCM REST API using the fxcmpy Python wrapper, we’ll first need to register a demo account, generate a token for authentication, and create a configuration file.  If you haven’t viewed our previous article on using the basic functionality of the fxcmpy Python wrapper (including generating a token and building a configuration file), it may be helpful to review that article first by clicking here.

Once we have our configuration file created and saved in the same directory as our current project, connecting to the API and pulling historical prices is a simple process.

First, we’ll create a connection object:

con = fxcmpy.fxcmpy(config_file='fxcm.cfg')

Next, we’ll use this connection object to pull daily candles for the GBP/JPY currency pair.  We’ll also use start and end dates so that we can define the period over which we backtest our strategy.  It’s important to note that the get_candles() method will return a DataFrame object, so we will store this in a newly created DataFrame that we will call “df”:

df = con.get_candles('GBP/JPY', period = 'D1', 
                     start = dt.datetime(2016, 1, 1), end = dt.datetime(2018, 6, 10))

That’s all it takes to get the historical data!  As a quick check, if you’re using a Jupyter Notebook you can quickly check on what we have stored in our DataFrame by simply typing out our “df” variable and executing the cell:

df

Defining the EMA Crossover Strategy Logic

Now that we have our historical data, it’s time to define the logic of our strategy. For this tutorial our EMA strategy will only place a single Buy order at a time with both entry and exit logic being controlled by the Exponential Moving Averages. Below is our strategy logic:

  • Entry Logic: When the Fast EMA crosses above the Slow EMA a buy order will be triggered. If there is an existing open trade no action is taken
  • Exit Logic: When the Fast EMA crosses below the Slow EMA the existing order will be closed, otherwise no action is taken

Now that we have our strategy logic defined in plain English, we can begin to build it out using code.

First, we will define our overall trade size and feed in an approximate pip cost to more accurately determine what our overall hypothetical profit/loss would be for the system:

pip_cost = .0879
lot_size = 10

Next, we will define the fast and slow exponential moving average values. For this tutorial we will be using an EMA Fast value of 12 days and an EMA Slow value of 20 days.

ema_fast = 12
ema_slow = 20

After defining the parameters of our strategy, we next need to populate our DataFrame with the exponential moving average values. To make this process even easier, we previously imported the Exponential Moving Average method from the pyti library and named it “ema”, which will now allow us to pass in historical prices that we pulled via the REST API and receive the Exponential Moving Average value for each historical price as a return.

For each of the lines below we will create a new column in our DataFrame (df[’ema_fast’]) and assign the return from the ema() method to the newly created column.  We will do this for both the ema_fast and ema_slow variables.

df['ema_fast'] = ema(df['askclose'], ema_fast)
df['ema_slow'] = ema(df['askclose'], ema_slow)

Now is a good time to review our DataFrame object to see what has changed by typing in “df” into a blank cell in our Jupyter Notebook and executing the cell to see the output.

df

As can be seen, we now have two new columns (which we created and named above) that have been populated with EMA Fast and EMA Slow values for each row.  One thing to note here is that we have ‘NaN’ for the first 11 rows of the “ema_fast” column and the first 19 rows of the “ema_slow” column.  This is not a mistake!  In order to get our first 12 day moving average we first have to have 12 values and the same logic holds for a 20 day moving average.

Now that we have both values we can compare the two columns to see when the ema_fast value is larger than the ema_slow value. When the ema_fast value is larger than the ema_slow value, we will populate a newly created column in the DataFrame called “position” with a value of 1, otherwise we will populate this column with a value of 0.

df['position'] = np.where(df['ema_fast'] > df['ema_slow'],1,0)

While the “position” column is useful for knowing when the fast value is above the slow value, it doesn’t tell us when the crossover actually occurs, which is when a signal will be generated. By examining the “position” column, we can see that a crossover occurs when the value changes from 1 to 0 or when the value changes from 0 to 1. By continuing to examine the “position” column, we can see that a change from 0 to 1 between rows corresponds to the ema_fast crossing above the ema_slow value, which matches our entry logic.

Our next task is to then isolate these changes, which we are able to do by applying the .diff() function to our “position” column. We will then store these values in a newly created column called “signal”. We will then print out our updated DataFrame to inspect the results.

df['signal'] = df['position'].diff()
df

What we can now see at a quick glance is that any row with a value of “1” in the “signal” column corresponds to a Buy signal according to our strategy logic, meaning we now have an algorithmic trading strategy that is generating trading signals in less than 20 lines of code!

Backtesting our Strategy

With our system now generating trading signals we can move on to backtesting the results. As a reminder, this backtest is designed to be quick and simple and, as such, does not reflect some important factors which include but are not limited to commissions, real-time spread costs, market impact, or slippage. Even if it were to take these items into account, no backtest can predict the future performance of a trading system as past performance is not indicative of future returns. As such, it’s always a good idea to forward test your strategy on a demo account or with small trade sizes on a real account before fully scaling up to your desired trade size.

To begin building our backtester, a quick and easy way to find rolling returns is to simply get the overall profit or loss for each trading day and storing it in our DataFrame “df”. To do so, we’ll simply get the difference between the open and close of the day and multiply this value by 100 to get the daily return into pips.

df['difference (pips)'] = (df['askclose'] - df['askopen']) * 100

Note:  We are using the value of 100 to convert the daily returns into pips because our system is currently trading a JPY based currency pair.  This value may need to be modified if you are trading different instruments.

Next, we will create a new column in our DataFrame called “total”. This column will store a running profit/loss amount for our strategy.  In order to populate this field, we will iterate through our DataFrame and, when we have a trading signal, we will add the daily profit/loss (converted to USD by using our pip_cost and lot_size variable) to the “total” column for each day that the signal is active. For all other days we will keep the total the same.

returns = 0
CountPL=False
for i, row in df.iterrows():
    if CountPL==True:
        returns += (row['difference (pips)'] * pip_cost * lot_size)
        df.loc[i,'total'] = returns
    else:
        df.loc[i,'total'] = returns
 
    if row['position'] == 1:
        CountPL=True
    else:
        CountPL=False

That’s it! By printing out the DataFrame “df” we can scroll through the values and see our daily equity change along with our running overall return. This allows us to not only view the P/L of each trade, but how much that P/L fluctuates on a daily basis. This gives us more information and clarity on how the strategy has performed over the backtested period.

Visualizing the Trading Signals and Returns

While data is always great to have, there are times when taking a step back and getting a visual picture can be helpful. Luckily, creating a visualization of the strategy returns is a simple task that only takes a few lines of code to complete.

First, we will create our chart area, chart size, and axis.

fig = plt.figure(figsize=(12,8))
ax1 = fig.add_subplot(111,  ylabel='GBP/JPY Price')

Next, we will plot out the daily market prices.

df['ask_close'].plot(ax=ax1, color='r', lw=1)

After this, we will then plot out our ema_fast and ema_slow moving averages.

df[['ema_fast','ema_slow']].plot(ax=ax1, lw=2)

Then, we will plot out each of open and close signals from our DataFrame.  As a reminder, when signal column has a positive value of 1 that will signify a buy signal. We will represent this on the chart with the “^” character. Additionally, when the signal column has a value of -1, this signifies a sell or close signal. We will represent this on the chart with a “v” character.

ax1.plot(df.loc[df.position == 1.0].index, 
         df.ema_fast[df.position == 1.0],
         '^', markersize=10, color='m' )
ax1.plot(df.loc[df.position == -1.0].index, 
         df.ema_slow[df.position == -1.0],
         'v', markersize=10, color='k')

Now that we have our signals visualized, we can next move toward plotting our returns.  To do this, we will create a new axis for our returns, create a new label, and then plot our returns.

ax2 = ax1.twinx()
ax2.set_ylabel('Profits in $')
ax2.plot(df['total'], color = 'green')

Finally, the only thing left for us to do is to show the chart that we’ve just created by using the following line of code:

plt.show()

That it! We’ve successfully displayed our results and can visually.

In Conclusion

As you can see, it’s quick and easy to build, backtest, and visualize the hypothetical returns for any given strategy with FXCM’s REST API and fxcmpy Python wrapper. In part 5 we will build a real-time strategy that executes live trades on a demo account. Click the link below to keep going!

Continue to Part 5


Risk Warning: The FXCM Group does not guarantee accuracy and will not accept liability for any loss or damage which arise directly or indirectly from use of or reliance on information contained within the webinars. The FXCM Group may provide general commentary which is not intended as investment advice and must not be construed as such. FX/CFD trading carries a risk of losses in excess of your deposited funds and may not be suitable for all investors. Please ensure that you fully understand the risks involved.