Cubesteak Central » MT4 Multi Broker Tick and M1 Data Project

MT4 Multi Broker Tick and M1 Data Project

Its been some time now, and I’d like to add an update. We have several folks logging ticks and are working on an automation system for distribution of the ticks. I hope to have something out and available very soon!

Thank you all for your patience and to those that are contributing, double thanks for your efforts!

Cheers to all,
CS

——-

Hello,

Thanks for stopping by. We have probably all used or at least heard of the Alpari databank, where M1 data from that broker is available all the way back to 2004. It has been very useful in back testing to achieve 90% modeling quality for back testing of EA’s.

The problem with this is that many people don’t use Alpari as their broker, nor do they wish too. So, this page is to help us collect this data ourselves.

Also, while M1 data is great for 90% modeling quality, it is still just a guesstimate that leaves much to be desired in the verification of EA’s. Paul from the MT_E_and_I Yahoo Group put together an excellent way to collect and use tick data for 99% modeling quality.

This page is to help organize MT4 traders in the gathering and dissemination of both Tick and M1 data from multiple FX MT4 based brokers.

I envision that individuals would be responsible for one, at most two different broker feeds for the majors. It would entail keeping them up to date and sending them to me at regular intervals (weekly, monthly, whatever people would commit to.)

I think it would be best if each broker was covered by at least two people, thereby reducing the risk of “gaps” in the data.

For my part, I would collect the data from those kind enough to participate, then aggregate it and make it freely available for download from a web page similar to this.

We could cover the more popular brokers:

IBFX
FXDD
NorthFinance
Tradex
VelocityFX
Neumex
FiboGroup
MIG

Or really, any broker / pair that people would commit to updating on a regular basis.

At this point, we are asking folks to cover the following pairs: EURUSD, GBPUSD, USDCHF, USDJPY, AUDUSD and USDCAD

The benefit of this is (I think) apparent but I’ll quickly state it anyway: the results will provide a growing cohesive databank of feed info that be turned into 90% or 99% modeling quality in back testing at any point for any covered broker / pair. While back testing isn’t perfect, this would go a long way to level the playing field, as it can provide the best possible data for YOUR broker!

If you would like to participate, please leave a comment on this page with which broker(s) you would like to cover. Additional comments on this project are also welcome. Please either leave your email address with the comment, or email me directly at cubesteak at gmail.com with how to contact you.

You can look at the current list of brokers and pairs to see what’s available.

Thanks very much in advance!

Cubesteak

38 Responses to “MT4 Multi Broker Tick and M1 Data Project”

cubesteak wrote a comment on October 28, 2006

I personally will cover demo servers of TradeX and a smaller CA based broker called FastFX for both tick and M1 data.

Bill Garrison wrote a comment on October 29, 2006

I can cover live and demo from IBFX for tick and M1 data.

Paul Hampton-Smith wrote a comment on October 29, 2006

Hi Cube,
great site. I feel we are at the beginning of a new era in backtesting EAs: thank you for recognising this and doing something real to promote it.

You might like to post some guidelines on which pairs to log. I personally have been logging 6 from FXDD: EURUSD, GBPUSD, USDCHF, USDJPY, AUDUSD and USDCAD

Paul

Emre wrote a comment on October 29, 2006

Interbank FX M1 here… Wiss start in the middle of the week….

Derk wrote a comment on October 29, 2006

I will contribute what I have: Demo and Live data from CFG.

I *could* do other brokers, but only with Demo, because I have no other live accounts…

cubesteak wrote a comment on October 29, 2006

Paul,

Excellent point – I’ll add the pair list to the front! Also, I’ll be sending out ftp address, etc. for everyone to start uploading what they have.

Thanks again to everyone!
CS

david kaziyev wrote a comment on October 29, 2006

I can collect data from MIG on eur/gbp

Lyle wrote a comment on October 30, 2006

I’d like to help. As of now, I just have Demo for IBFX…but that may soon change.

I also would be able to help analyze the resulting data files.

Lyle

Wally wrote a comment on November 1, 2006

Hi Cube,

I just kicked off the ticlogger on FXDD for the pairs you listed. I’ll keep it running on both live and demo accounts as long as my machine keeps running and my connection stays healthy!

Wally

cubesteak wrote a comment on November 1, 2006

Thanks everyone for all your help! Here’s to a new era in back testing in MT4!

dusktrader wrote a comment on November 1, 2006

I too will follow your instructions to start collecting from IBFX (Metatrader running on a server here, with a commercial fiber connection to the net — no flaky consumer bandwidth)…

I’m assuming it would be helpful to have multiple sets of data from the same sources, as some could have gaps in them.

I do some programming on the side and could probably help write some code to merge the data into cohesive groups, highlighting the discrepancies… if that would be helpful. Drop me a line.

cubesteak wrote a comment on November 1, 2006

Hi Paul,

Thanks very much for the offer! There is a good chance that we’ll take you up on that.

Cheers,
CS

shimon doodkin wrote a comment on November 11, 2006

you can put it to be downloaded from upload free space services
like http://www.4shared.com/

bwilhite wrote a comment on November 14, 2006

Cubesteak,

This looks like a great project and I think I would like to help…let me determine which brokers and I’ll get back to you. A couple of suggestions…I notice that the tick data collects only bid data, maybe we should collect ask data too…I could think of a few reasons why which I won’t go into. It might also be a good idea to find a way to time-stamp the data in a manner so that we can compare it broker to broker…maybe pull the time off the computer system clock…I’ll see if I can modify the EA to do this today. I’ll get back to you on data collection. Cheers!

bwilhite wrote a comment on November 14, 2006

Cubesteak,

I just wanted to let you know I successfully modified the EA to write bid and ask, as well as take time-stamp from local system clock. If everyone who collected data then set their system clock to GMT it would be a simple matter to match up the broker data. This would help in backtesting. Just a suggestion. I can get modified EA to you, if you want it.
Something else I noticed. I was trying to collect data on 3 different pairs, two different brokers…and I’m not sure all data is being collected in this instance. Even collecting just 3 pairs on one broker might miss some data. I’m fairly sure that this is happening and I believe it is a result of the way the EA works. I’ll be working on the problem for my own purposes this week. Let me know if this could be helpful.

Thanks.

Blackday wrote a comment on November 28, 2006

How can I contact you Cubesteak? I can get tick data for these pairs:-

EURUSD
GBPUSD
USDCHF
USDCAD
USDJPY

PS RSS says ‘Address type not known or supported’ (Opera)

avi wrote a comment on December 1, 2006

gbp/usd data tick

Nikolay Kolev wrote a comment on December 7, 2006

I would like to participate. I’ve already started to log ticks from FXDD for EURUSD, GBPUSD, USDCHF, USDJPY, AUDUSD,USDCAD pairs.

Matt wrote a comment on December 16, 2006

Is any data available yet? I would love to use it to research past news events….

thanks,
Matt

cubesteak wrote a comment on December 17, 2006

Hi Matt,

Not yet, sorry… Still working on how best to distribute these big files in a way that is easy for everyone to use, etc.

Soon though, I promise!

-CS

Matt wrote a comment on January 10, 2007

Let’s all distribute it using distributed file sharing. I use it all the time, easy and works great.

http://torrentfreak.com/how-to-create-a-torrent/

Ruby wrote a comment on January 13, 2007

We will collect Neuimex live Tick + M1 data for 6 Majors. Please send me by email an explenation + username / password.
Ruby

Scripes wrote a comment on February 2, 2007

Hey! I would like to join you guys!

FANTASTIC enforcement!

I’ll start my demo account on FXDD and will capture the 4 majors.

Is there any systematization? Did you guys divide shifts, pairs, days, weeks, brokers, servers? Or people are randomly getting the data at their will?

How would I give my fxt files to you guys? How would I get the past data I didn’t cover?
perhaps we could distribute it through Rapidshare.

History wrote a comment on March 15, 2007

Did you check standard M1 history from 1999 in MetaTrader terminal?
History Center (F2) – > choose symbol -> Download.

cubesteak wrote a comment on March 15, 2007

Hi History,

Yes, I have indeed seen that, and the M1 portion of this project is a lot less important, but still relevant.

The history center data is very generic and not really representative of what you can actually trade against at your broker.

This project is to get broker specific tics so that we can know without a doubt how something would have performed.

That said, I’ve been distracted by a new job lately, so I haven’t been able to post the data that folks have been uploading.

To all, I apologize for that… I hope to change that soon.

-CS

Wayne wrote a comment on May 7, 2007

What about IBFX Mini account data?
The IBFX Mini accounts all end in m as in EURUSDm is anyone collecting this data or are Mini accounts not an issue at this stage?

Viktor wrote a comment on July 19, 2007

Guys, How can i download tick M1 data for you guys. please let me know.. thanks

John wrote a comment on July 24, 2007

I can’t wait, I need data to test a maddening system that’s going great on backtesting data from Alpari and metaquotes servers but hopeless in forward testing on demos, really need several brokers data to test.

Only interested in EUR/USD but would be happy with every brokers test data you got on it. (maybe it works at some not at others)

Gmail can take up to 10MB attachments I think.

Thanks a million, maybe literally.

emeric wrote a comment on October 24, 2007

I would be interested to participate.

How should I start for downloading data? Seems id and password are needed.

I have some datas for ibfx and fxdd livbut not a so long history…

gOdFeLLa wrote a comment on January 8, 2008

Can’t wait for the data. being waiting so long for a decent good tick data

MrH wrote a comment on January 19, 2008

Hi,
this logging should really not be done from desktop computers, but on a VPS (virtual private server), with restarts in the weekends to ensure total and complete capture of all data and to avoid typical combination problems. Why not rent a VPS, run like 20 different demo versions of 20 different brokers and save all data. The difference between demo and live (requotes and slippage) is only something you can solve with forward testing anyway. I’d be willing to chip in a couple of $’s per month to get the data. Combination of all data shouldn’t take more than 10 minutes per week and the admin could take care of that (Cubesteak?).

victor wrote a comment on March 29, 2008

whats happened to the tick project? is it still running?

Kevin wrote a comment on May 17, 2008

Hi. I’d like to join in on the tick data collection. I have 2 dedicated servers with 99% uptime that I can use for this. Please let me know how I can start and what brokers you want me to do. Thanks!

180 tramadol on wrote a comment on September 20, 2008

on tramadol 180 180 tramadol by

jok wrote a comment on October 23, 2008

yes brokar project

George wrote a comment on November 12, 2008

I would like to participate but, I’m interested in doing GBP JPY for 4 Hour TF on IBFX Live account.

InterbankFX-MT4 Mini Accounts 3
I’m interested in this project however it looks like it died off.