Before we can delve into technical analysis, you must first understand the underlying data in our database. I must warn you that while this chapter is somewhat dry & boring, it is critical you read and understand it.
Unlike a common stock or fixed income instrument where you can easily pull years of historical price and volume data, futures contracts must be chained together in order to create a "Continuous Contract" that we can then use for historical analysis purposes.
Remember, each future contract is a fixed contract for a specific month and year. Each one will be priced differently and the difference in pricing between these contracts is referred to as "Contango" or "Backwardation" depending on if the front month is higher or lower than the back month . Therefore you must determine a method for connecting all of these various contracts to present them in a continuous chain for analytical purposes.
To reiterate this point, lets us look at Crude Oil. West Texas Intermediate Crude, otherwise known as CL, has 12 contract months each year - So if you want to view a 30 year chart you have to chain 30 years * 12 months = 360 Crude Contracts in order to create the continuous contract.
We employ (multiple) third party data providers to source the Continuous Raw Data into our database. Our batch then runs, updating all calculations, reports, charts, and data tables.
Our Daily Batch run is typically ready 9:00pm EST each evening.
Our COT batch run is typically ready after Midnight EST each Friday night
There are multiple data providers with a vast array of available data, robust delivery technology, and responsive support teams. As there is so much information, it certainly isn't cheap to source the data. You will be charged anywhere from $100 per month for lesser quality data, to $1k or more per month for high quality data.
If you want to source raw unchained data from the exchanges, it can cost up to 3k per contract! Then, you have to build out the technology for chaining and maintaining the data - Certainly not a viable option for the individual or small office business.
As a trader you will always be looking for an edge, and looking to control your costs. As a company we are happy for you to explore other data providers' offering, as we believe we have a compelling product offering and benefits that will tilt the scales in our favor. We don't believe you will find a better price, or a cleaner or more robust data source. There are many other differences to consider in terms of what you are getting for your buck.
What you get from us that you wont see with competitors:
- An additional layer of data validation and checks on the underlying data.
- Custom analytics and indicators in Heat Map format.
- Statistical Odds tables along with Charting application
- Access to the 3PC blog with commentary and videos.
Due to the amount of data involved in maintaining a historical database spanning many decades, there are going to be occasional gaps and errors. While this is a unfortunate fact of life, it simply isn't worth your time, effort and money to attempt to create these continuous contract chains yourself.
We do have additional checks in house to try to find and correct errors from the vendors - and this is on top of the many months we spent scrubbing the data & testing the platform. So we are providing the most accurate data we can, but if you spot something we missed, please send a note to our tech-support team & we'll do our best to remedy the error.
Let us now get back to understanding the chaining methodology employed in our platform.
Chaining Methods & Useage
When you are trading futures contracts one of your primary concerns is liquidity. As a future contract nears expiration, traders will roll their position into one of the "back" contracts. So for example, if you are trading May Crude Oil, as you near expiry traders will roll into June. So this is called "Open Interest Switch or Liquidity Based Roll" and it is the method we use for actual trading; and it is therefore the roll date we use for our data in all reports and charts.
You have the option of implementing a price adjustment, which is called the "Backwards Adjustment". You shift successive contracts up or down by a constant amount so as to eliminate jumps, working backwards from the current contract. The price of the current continuous contract will be "true" and match market prices; however, you recalculate your entire history on every roll date. And it is this last statement that causes us an issue when looking at historical data using this method.
Each time you roll the future, the entire Price history changes by the price shift used to eliminate the jumps caused by Contango/Backwardation - This means when you look at 2015 or 2000 or 1980 data, it has been "shifted" hundreds of times, and the end result is showing you prices that never happened.
To illustrate this point, lets look at this link which shows a weekly chart of Soybeans using the Backwards adjust method. This is what you see in many public websites, as well as paid charting applications.
What you can see is:
- Present day prices matching the close price of the most actively traded contract (904 1/2 as of this writing). This is what we want - So far so good.
- Lets look at 2008 data however - We can see on this chart that in Dec 2008, price hit a low of NEGATIVE -133 ! - I can assure you this never happened, so this data point is useless to us.
Because of this skew to the data, we have elected not to back adjust, so there are price gaps embedded on each roll date. To smooth the gap we roll the contract over a period of days so as to minimize the gap upon roll.
This is a eloquent compromise in our view because A) You always have correct current data - mimicking the actual contracts you trade. B) Our historical prices match exactly, except slight differences around the roll period. C) No outrageous skew from back adjusting data which would skew all of our historical analysis.
So in conclusion, for present day trading, we are using "Open Interest Roll or Liquidity Roll" - This means we are always in the most actively traded contract. Also, we do not back adjust price in order to not skew history. To minimize gaps we roll over a multi day period.