Know Your Benchmark

A recent TABB Group article provided a great introduction to the futures execution algorithm landscape, including the history of the space, varied requirements of different industry users, buy vs. build argument, and a general outlook on the most popular types of algorithms. A common thread throughout the piece is the importance of transaction cost analysis (TCA) and its usefulness in analyzing trading performance and aiding the trader/portfolio manager in tweaking algorithm behavior or unearthing a need for outright customization. Understanding execution costs is a multi-step process, and a proper understanding of TCA metrics provides useful insights that can be fed back into the trading process and further improve trading performance.

For example, in the graphic below, we illustrate the market’s price path along with a theoretical buy order being filled and identify different potential costs associated with an order that is not apparent when simply looking at average fill prices. The opportunity cost related to differences between signal generation and execution time, market impact costs related to demanding liquidity from the market, and the microstructure costs associated with the effectiveness of the trader are examples of more granular levels of cost breakdown that can help inform trading. Whether a trader is crossing the spread too quickly, demanding too much liquidity from the market, or waiting too long to execute orders are issues which may require different execution solutions.


TCA metrics can be broadly categorized as price-based or volume-based. Price-based metrics typically represent a measure of slippage from a defined benchmark; volume-based metrics tend to represent the ratio of an order’s volume against things such as the average daily volume and interval volume, or more granular aspects such as the passive fill rate.

Price-based Metrics

Price benchmarks set a reference point from which traders can evaluate their average fill price when analyzing their execution costs. Common price benchmarks include things such as arrival price, open price, close price, and volume-weighted average price (VWAP). Interpretation of metrics using these benchmarks differs based on whether a benchmark is pre-trade, post-trade, or intraday.

Pre-Trade Benchmarks

Pre-trade benchmarks, such as the previous closing price or the opening price, may be used to estimate a timing or opportunity cost. Price drift from the time a portfolio manager makes a trading decision, often based on prior day close prices, to the start of trading, represents alpha decay that an execution strategy is not able to capture. This is the first ingredient of overall implementation shortfall and can be measured as the difference in arrival price slippage vs prior close slippage. Significant opportunity costs indicate that time could be well spent trying to identify a trading signal sooner, or delivering the resulting order to the market quicker.

Arrival price, the price in the market at the time an order is entered, is a particularly important pre-trade benchmark[1]. Slippage metrics versus arrival price provide a robust metric for an algorithm’s performance, as it should encompass implicit and explicit costs – both the cost of demanding liquidity, alpha decay over the trading horizon, and total market impact.

Intraday Benchmarks

Various VWAP measurements represent what the weighted average price in the market was during different time intervals. ‘Day VWAP’ measures this average over the entire trading day; ‘interval VWAP’ over the lifetime of an order; and ‘end-of-day VWAP’  the average between the arrival time and the end of the day, which with a large sample size can be used to interpret whether a trader’s market timing is contributing positively or negatively to performance. Slippage from ‘interval VWAP’, the most commonly monitored performance benchmark, represent the cost of demanding liquidity and the quality of child order placement strategies. It is a less comprehensive cost measure than arrival price slippage, but tends to be less volatile between individual orders and provide a cleaner performance view to traders with long horizon strategies.

Analyzing slippage with regards to VWAP requires some caveats. VWAP slippage is not very comparable across instruments due to differences in volatility. Variance in VWAP slippage between individual orders will be driven mostly by whether the execution strategy is pursuing volume driven order placement (as is the case with VWAP and POV algorithms), and if so whether the algorithm’s volume forecasts line up well with the trading day’s actual volume. VWAP can also be heavily skewed by a large market participant. Traders, institutional or otherwise, who make up a large percentage of the volume relative to the VWAP metric they are using as a benchmark may effectively set the VWAP; in such cases, the market impact caused by their own trades reduce the information provided by VWAP in more normal circumstances.

Participation weighted price (PWP) is another intraday benchmark with multiple variants. In this case, the set participation rate is the variable; for example, a 10% PWP is the volume weighted average price that would correspond to the trader’s order making up 10% of the volume following the arrival of the order. Various PWP metrics can be analyzed alongside each other, provided the sample size is appropriate, when trying to determine whether trading horizon should be adjusted.

Post-Trade Benchmarks

Post-trade benchmarks include the close or settlement, next day open, and next day close.  The difference in arrival price slippage and post trade slippage can also be used to help analyze the difference in temporary market impact and permanent market impact. Large slippage versus end of day metrics may lead a trader to conclude that they can reasonably trade more passively. Some market participants also focus on close price slippage because their investment objectives are tied to end of day pricing benchmarks, or they may be sensitive to end of day mark to market ramifications. Participants trading significant volumes during short periods near the market close may materially affect the final price, depending on settlement procedures. In these instances, like the problem posed by large percentage participants when comparing against a VWAP metric, a participant may end up effectively setting the settlement price; a trade may look like it achieved a price better than the settlement benchmark, but it may have incurred substantial market impact.

Volume-based Metrics

Market participants should monitor volume-based metrics alongside price-based metrics when analyzing trading performance and execution quality. Volume metrics help put slippage metrics in context; high participation rates correlate to higher arrival price slippage. Volume metrics also help participants analyze capacity constraints of their strategy and optimize the duration of their orders.

Participation rate metrics include: percentage of daily volume, percentage of average daily volume (ADV), and percentage of interval volume. Elevated figures in these metrics would generally indicate a higher cost of market impact during the respective trade; slippage versus arrival price tends to be higher than normal while slippage versus VWAP becomes muted (high participation rates cause the trade to contribute more heavily to the VWAP).

The passive fill rate may be used to determine how adept an algorithm is at avoiding crossing the spread when executing. High passive fill rates may indicate a level of success in avoid avoiding crossing the spread and minimizing the cost of demanding liquidity. However, a high passive fill rate by itself does not indicate high quality execution – passive fill rates can also indicate buying into a falling market, and can reflect adverse selection in market timing, as waiting for a passive fill can cause a trader to execute at a less desirable price as it trends away.

The relative performance measure (RPM) is another metric, particularly useful for comparing trades across products and time. Generally, RPM represents the percentage of trades executed at a worse price than the trader’s fill price during the execution. Whereas certain price metrics may not be readily comparable, RPM provides a simple, robust metric that a trader may use to compare executions. This can be extended to monitor consistency of trading performance, as this metric is less sensitive to volatility differences between products. Inconsistent RPM statistics could highlight potential trading issues warranting further investigation.


The Whole Picture

Transaction cost analysis is a process. No single metric should dominate the analysis, and users should educate themselves on what different combinations of metrics could mean. Traders shouldn’t miss the forest for the trees; ignoring how these different metrics interact could mean missing out on potential improvements to their overall trading process. Some questions you should be able to answer from a composite of different TCA reports include:

  • How often is my trading signal correct, but I am not able to put on a position due to a limit price?
  • What was the price activity during the life of my order?
  • What do my percentage of volume statistics look like? Am I impacting the market too much?
  • Is the slippage I’m experiencing acceptable with regards to my trading goals? Is it consistent with my back testing and simulation assumptions?
  • Am I comparing metrics appropriately? Is there something obvious that I can work on improving?

RCM-X works with its execution clients to review their transaction cost analysis and make actionable recommendations based on the trade data. Contact Mike Aufmann at 312-870-1537 for more info on how RCM-X can assist with your execution and transaction cost analysis needs.

The performance data displayed herein is compiled from various sources, including BarclayHedge, and reports directly from the advisors. These performance figures should not be relied on independent of the individual advisor's disclosure document, which has important information regarding the method of calculation used, whether or not the performance includes proprietary results, and other important footnotes on the advisor's track record.

Benchmark index performance is for the constituents of that index only, and does not represent the entire universe of possible investments within that asset class. And further, that there can be limitations and biases to indices such as survivorship, self reporting, and instant history.

Managed futures accounts can subject to substantial charges for management and advisory fees. The numbers within this website include all such fees, but it may be necessary for those accounts that are subject to these charges to make substantial trading profits in the future to avoid depletion or exhaustion of their assets.

Investors interested in investing with a managed futures program (excepting those programs which are offered exclusively to qualified eligible persons as that term is defined by CFTC regulation 4.7) will be required to receive and sign off on a disclosure document in compliance with certain CFT rules The disclosure documents contains a complete description of the principal risk factors and each fee to be charged to your account by the CTA, as well as the composite performance of accounts under the CTA's management over at least the most recent five years. Investor interested in investing in any of the programs on this website are urged to carefully read these disclosure documents, including, but not limited to the performance information, before investing in any such programs.

Those investors who are qualified eligible persons as that term is defined by CFTC regulation 4.7 and interested in investing in a program exempt from having to provide a disclosure document and considered by the regulations to be sophisticated enough to understand the risks and be able to interpret the accuracy and completeness of any performance information on their own.

RCM receives a portion of the commodity brokerage commissions you pay in connection with your futures trading and/or a portion of the interest income (if any) earned on an account's assets. The listed manager may also pay RCM a portion of the fees they receive from accounts introduced to them by RCM.

See the full terms of use and risk disclaimer here.