Expand Data Analysis and Insights with Space-Time Benchmarking

by   |   July 29, 2016 2:45 pm   |   0 Comments

Raul Valdes-Perez, CEO and co-founder, OnlyBoth

Raul Valdes-Perez, CEO and co-founder, OnlyBoth

In sports, “most improved” designations are insightful and motivating. Such designations imply not just a comparison to other players, but also to the same player in the recent past.

Outside of sports, peer comparison is prominent in business, where it’s known as benchmarking. The benchmarking task traditionally has been carried out manually (and mentally). As a result, the inability to deal with too much data has led to a focus on peer comparisons during a fixed measurement period, e.g., last month, quarter, or fiscal year, as well as other shortcuts. Mixing in the past is a lot of extra work.

Given the presence of lots of data and metrics, hard mental work, few experts, and necessary shortcuts due to these and other factors, benchmarking can be enhanced with artificial intelligence. Consider, for example, how artificial intelligence has changed web searches. Compare the effectiveness of a modern web search, using search engines infused with artificial intelligence, with the browsable directory of web pages that marked the early days of Yahoo.

Automation through artificial intelligence can give rise to an expansive role for business benchmarking, which we’ll call space-time benchmarking, because it enables comparisons to peers (who occupy other spaces) and back in time, and not just considering improvements but changes in general.

After all, to promote change and improvement, why not benchmark on changes and improvements?

Space-time benchmarking is impractical without automation because the scope for benchmarking insights increases by more than double. For example, consider that any metric X can be combined with change-in-X to enable insights such as “Acme Inc. had the smallest quarterly rise in revenue of all companies that have at least $1 billion in revenue.”

A Space-time Benchmarking Model

Related Stories

To Start an Artificial Intelligence Strategy, Follow the Data.
Read the story »

Improve Automation in the Enterprise with Augmented Intelligence.
Read the story »

How to Improve Machine Learning: Tricks and Tips for Feature Engineering.
Read the story »

How Shutterstock Uses Machine Learning to Improve the User Experience.
Read the story »

Let’s do a simple model of the effects. Suppose there are N metrics and M possible peer groups. For a given target entity, simple benchmarks imply NxM amount of work. Now let’s throw in change-in-X metrics. The work doubles to NxM + NxM = 2NxM. Further allowing pairings of X with change-in-X, the work goes up to 2NxM + NxM = 3NxM. If we consider both quarterly changes and, for example, annual changes, then it’s 4NxM. If change-in-X can be paired with change-in-Y, as in, “The smallest annual revenue increase of anyone whose labor force grew by at least 10 percent,” then the work expands still more. The total impact is actually greater in reality, due to combinatorial effects.

Space-time Benchmarking on Public Data

We have carried out automated space-time benchmarking experiments on several public data sets. The most recent application benchmarks the tax systems of 195 countries using the USAID Collecting Taxes Database. The same source makes historical data available, so that metrics such as corporate income tax productivity has values both for the last measurement period and for three years earlier. Here are three example outputs, discovered and written by software, on the United States that involve changes over time:

  • The United States had the sixth-biggest rise over three years in tax-collection overhead (+38 percent) among all the 195 nations. That 38 percent increase compares to an average 16.2 percent increase and standard deviation of +87.5 percent across the 195 nations. Also, that 38 percent increase represents a rise from 0.45 percent to 0.62 percent.


  • The United States had the seventh-biggest rise over three years in corporate income tax collection as a percentage of GDP (+190 percent) among all the 195 nations. That 190 percent increase compares to a median of 0 percent across the 195 nations. Also, that 190 percent increase represents a rise from 0.9 percent to 2.6 percent.


  • The United States had the biggest rise over three years in corporate income tax productivity (+130 percent) among the 28 nations with at least a 35 percent corporate income tax rate. That 130 percent increase compares to an average 1.7 percent decrease and standard deviation of +46.6 percent across the 28 nations. Also, that 130 percent increase represents a rise from 0.03 to 0.07.


Enter a country and browse the insights yourself at taxes.onlyboth.com.

Automated benchmarking relies on a classical AI approach:

    • Study the potentially large space of acceptable solutions


    • Figure out what makes one solution better than another


    • Devise an algorithm that will search this space while using knowledge to guide its search in favorable directions


    • Rank the outputs according to other domain knowledge, and


    • Write up conclusions as a human consultant would


Space-time benchmarking across peers and time greatly expands the potential to find valuable and motivating insights. It places the task even further beyond the reach of traditional, largely manual methods, just as full-text search, rather than just search over titles, took search engines far beyond what people could accomplish without automation.

To promote change, let’s benchmark for it.

Raul Valdes-Perez co-founded OnlyBoth in March 2014 and is now the CEO. He co-founded Vivisimo in 2000, a search software company that provided enterprise products and web-based consumer services, serving as its CEO for nine years and as Chairman for twelve until its acquisition in 2012 by IBM. At Vivisimo he was named a 2007 Ernst & Young Entrepreneur of the Year for the North Central Region and a top ten reader favorite for Entrepreneur of the Year by Inc. magazine. Earlier, he was a member of the Carnegie Mellon computer science faculty. He received a computer science Ph.D. from Carnegie Mellon in 1991.

Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise, plus get instant access to more than 20 eBooks.

Tags: , , , , ,

Post a Comment

Your email is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>