A Monte Carlo Simulation function for your back-test results – in R

In this post on bettersystemtrader.com, Andrew Swanscott interviews Kevin Davey from KJ Trading Systems who discusses why looking at your back-test historical equity curve alone might not give you a true sense of a strategy’s risk profile. Kevin Davey also writes on the topic here for futuresmag.com. So i wrote a Monte Carlo-type simulation function (in R) to see graphically how my back-test results would have looked had the order of returns been randomized. I use ‘n’ to denote the number of simulations i would like to do, and exclude the highest and lowest 5% of return streams to form the blue ribbon. For the middle red ribbon i use the middle 50% of return streams (ie. excluding the 1st and last quartile). The second and last parameter (default TRUE) is a boolean which if TRUE will run the sampling procedure in R with replacement, and without replacement if set to FALSE. See this link if you need an intuitive explanation of what sampling with and without replacement means. Below is an example of the output for 10k simulations, with and without replacement for a random set of returns on a hypothetical strategy spanning 7 months. The black line is the actual historical equity curve. Whether or not the return distributions satisfy your risk appetite will come down to a more refined drawdown analysis together with the host of other risk metrics available for consideration. At least with the function below you have the stats in a dataframe and a graph to start…the starting point for any statistical analysis.

mcsimr_without_repmcsimr_with_rep

I read the data into R using the read.csv function. You can use the same data here (simSample.xlsx), just be sure to change the file type to CSV first, as WordPress only allows for storing .xls and .xlsx Excel files. Below is the source code to add the function to your environment. I add the required packages then create the mcsimr() function to do the import, manipulate the data and build the graph using ggplot. Remember to set your working directory as the directory in which you have stored the simSample.csv file.

If you would like to run the simulation 1,000 times without replacement, then the function should read:

mcsimr(1000, FALSE)

If you would like to run with replacement, include TRUE for the second argument or leave it blank, as the default is TRUE.

mcsimr(1000)

Ok, below is the full source code

I am aiming to have the function code on github soon [EDIT: The above is hosted (with love) by GitHub, however enhancements are still a work in progress], hopefully with some enhancements allowing the user to specify an xts object for the raw data (ie. backtest results) as well as letting the user decide which percentages to use for the bands. Some drawdown stats would be useful too. I have built something which incorporates a drawdown analysis of every ‘n’ return stream, however, it needs some more work to speed it up as you can imagine 10k xts objects with 500+ of daily return data could take some time to analyse individually.

Lastly, i wonder if something like this could fit into the QuantStrat or TradeAnalytics packages…or if there already is something similar do let me know. [EDIT: Since publishing this post Brian Peterson (maintainer of multiple Finance R packages like ReturnAnalytics, quantmod and Quanstrat) got in touch and asked if i would work with him in getting this function into the blotter package, a dependency of Quantstrat. Brian has also suggested a few improvements to the function including using Block Bootstrap to account for autocorrelation and plot.xts as opposed to ggplot for faster graphical rendering. So look out for an updated post on that.]

Happy hacking!

 

Advertisements

8 thoughts on “A Monte Carlo Simulation function for your back-test results – in R

  1. Jasen:
    Thanks for sharing this A Monte Carlo Simulation writing.
    Your listed source code has many "
    Possible needs to clean up on this.
    It is a great writing for Monte Carlo Simulation in R.

    Like

  2. Thanks, glad you like it. I am actually working with the brilliant team who maintain Blotter/Quantstrat on getting it included in that package. Of course, this only helps if your BT is run in that package, which i highly recommend.

    That funny formatting is likely to do with how HTML renders R code and i missed it after editing a little punctuation error in the original post. This happens even though the source code is inserted inside the “[sourcecode language="R"]” tag. Its also why i use “=” as opposed to “<-" for assignment. Nevertheless, i finally got round to correcting it. Thanks for reading.

    Liked by 1 person

  3. Thanks for sharing with good works.
    Since your R codes using chart.CumReturns and cumsum to calculate the numbers and format the chart.
    There are some (subtly) different between these two functions. If read plots if you can see the different from ggplot and chart.CumReturns. There is different on the results (black line) – geometric chaining and arithmetic chaining (a sum).

    Like

    • Right you are…good spot. As far as i see it i have 2 options, 1. Set the geometric argument to FALSE in the chart.CumReturns() function or 2. Use log-adjusted returns. Let me know your thoughts by getting in touch via the Contact page.

      Like

  4. Pingback: Best Links of the Last Two Weeks and A Shout-Out to @QuantNews | Quantocracy

  5. Pingback: Block Bootstrapped MC function for backtest results – in R | OpenSourceQuant

  6. Hello,
    first of all, thanks by your great code, help me a lot.
    Maybe it is a dumb question, sorry about that, but you got a 95 percetil (95_ret) in mc simulation, why you display 90% on the chart?

    Thanks again

    Like

    • Hi Janderson, glad you finding it useful. The idea of the chart is to show the middle 50% and 90% of the simulations…so it requires isolating the 25th to 75th percentiles and the 5th to 95th percentiles respectively.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s