News & Features

Share:

The Tennis Recruiting ranking system provides a predictive measure of junior tennis players' abilities. In other words, the ranking system can be used to predict winners - and probabilities for upsets - in match-ups between players.

For example, consider the 2,616 singles matches at the 2015 Clay Court Championships. In this tournament, the player ranked higher by Tennis Recruiting defeated the lower-ranked player in 2,020 matches - meaning that Tennis Recruiting correctly predicted 77.2% of all matches. Our system also called for 561 upsets - which was within 1.4% of the 596 upsets that actually occurred.

So... is 77.2% a good number? And is this tournament representative of how the system would perform in other tournaments?

This brief article tries to answer those questions. We take a look at how the Tennis Recruiting (TRN) system performs when compared against other junior tennis ranking and rating systems. We compare the performance of our system with Universal Tennis Ratings (UTR), USTA Points Per Round (USTA), and ITF Junior Rankings (ITF) for a set of sectional, national, and international tournaments.

Rating Comparison Systems

To evaluate these ranking and rating systems in terms of predictiveness, we record the rankings of the players according to each system before a tournament starts, and then we count the number of matches in the tournament that the systems get right and wrong. For example, if a ranking system gets half of the matches right, then the resulting percentage is 50%.

The following table shows how the four rating systems performed for a set of recent high-profile tournaments. Each row corresponds to a single tournament, and each column corresponds to a different rating system. The table itself displays the percentage of matches where the higher-rated player defeated the lower-rated player. The tournament names link to heatmaps and forecasts performed by Tennis Recruiting during the tournament. The percentages link to post-tournament ranking analysis for the corresponding system.

Date	Tournament Name	Matches		Wins Predicted
				TRN	UTR	USTA	Other
12/27/14	2014 USTA Winter Nationals	949		75.3%	74.7%	69.5%	—
4/04/15	2015 Easter Bowl	831		75.5%	71.7%	66.7%	71.1%
5/31/15	2015 Roland Garros Junior French Open	124		71.8%	65.7%	—	57.4%
6/05/15	2015 Eastern Sectional Championships	682		81.2%	78.3%	74.9%	—
6/06/15	2015 Florida Sectional Championships	914		80.1%	78.9%	69.4%	—
6/12/15	2015 Texas Grand Slam	1,768		80.5%	81.1%	71.0%	—
6/13/15	2015 Southern Sectional Championships	2,346		79.9%	78.3%	67.8%	—
6/17/15	2015 Mid-Atlantic Sectional Championships	601		82.7%	81.3%	70.5%	—
6/20/15	2015 Midwest Sectional Championships	1,919		82.7%	82.1%	74.0%	—
6/20/15	2015 Southern California Sectional Championships	845		79.5%	79.0%	68.0%	—
7/04/15	2015 Wimbledon Jr. Championships	126		70.6%	68.7%	—	58.7%
7/11/15	2015 USTA Clay Court Championships	2,616		77.2%	75.0%	68.5%	—
8/01/15	2015 USTA National Championships	2,627		76.3%	75.1%	69.6%	—
8/31/15	2015 US Open	358		72.9%	70.0%	—	69.2%
12/27/15	2015 USTA Winter Nationals	2,172		76.0%	75.4%	68.8%	—
1/23/16	2016 Australian Open	374		76.2%	73.0%	—	71.1%
4/02/16	2016 Easter Bowl	815		72.5%	70.1%	65.5%	63.0%
5/25/16	2016 NCAA Championships	187		75.9%	72.2%	—	72.8%
5/29/16	2016 Roland Garros	376		73.9%	74.5%	—	68.5%
7/02/16	2016 Wimbledon Jr. Championships	371		77.1%	73.6%	—	72.0%
7/17/16	2016 USTA Clay Court Championships	2,618		75.9%	74.6%	67.2%	—
8/05/16	2016 USTA National Championships	2,617		76.8%	75.3%	69.3%	—
8/29/16	2016 US Open	363		73.3%	70.1%	—	69.4%
12/27/16	2016 USTA Winter Nationals	2,134		76.5%	74.4%	69.0%	—
1/16/17	2017 Australian Open	370		70.8%	71.4%	—	68.5%
7/2/17	2017 Wimbledon Jr. Championships	365		72.6%	70.0%	—	67.1%
7/17/17	2017 USTA Clay Court Championships	3.072		76.0%	75.7%	67.5%	—
8/5/17	2017 USTA National Championships	3,079		74.9%	75.4%	67.7%	—
9/2/17	2017 US Open	642		68.4%	66.1%	—	61.3%
12/27/17	2017 USTA Winter Nationals	2,160		75.0%	73.4%	67.6%	—
Totals:		38,421		76.8%	75.6%	68.6%	67.2%

As you can see in the table, Tennis Recruiting outperformed all other rating systems considered in all but four of the tournaments we studied in this analysis (from the 2014 USTA Winter Nationals through the 2017 US Open).

Focusing on the 2015 USTA Clay Courts, we noted above that TRN correctly predicted 2,020 matches - or 77.22%. By comparison, the UTR and USTA correctly predicted 75.0% and 68.5% matches, respectively. (We do not show numbers for the ITF because few players in the Clays have ITF rankings.)

Looking across all tournaments in the table between December 27, 2014 and August 1, 2015, notice that TRN accurately predicted almost 79% of all matches - which is superior to all other systems considered.

Notes on the Evaluation

There are several technical notes regarding the evaluation:

• For the purposes of this analysis, we define an upset to be a match where a lower-rated player defeats a higher-rated player - regardless of the rating difference.


• Defining an upset in this way is different from the study put out by Universal Tennis on the 2014 Winter Nationals. In that study, Universal Tennis defines an upset to occur when a player loses a match to an opponent rated 1.0 or more points below. While that approach is certainly meaningful, using ranges like this is somewhat arbitrary. For example, at Tennis Recruiting, we could define an upset as a loss to someone more than one star rating below (e.g., a 4 Star losing to a 2 Star). The larger the range, the fewer the number of upsets. For this reason, we use the ratings to come up with a complete ordering of all the players - and we define upset at the finest level for all systems.


• For each tournament we studied, we recorded TRN, UTR, USTA, and ITF ratings for the competitors immediately prior to the tournament. New rating values were not calculated or considered during the tournament. Only the rating values prior to the tournament are used for evaluation.


• A match is counted only when both players have ratings.


• Every algorithm treats defaults, withdrawals, walkovers, and retirements differently. For simplicity, we do not include these matches in this analysis. Only matches played to completion are included here.


• ITF rankings are only considered for ITF tournaments because the majority of players in USTA tournaments do not have ITF rankings. Likewise, we do not include analysis of USTA rankings for international events that have only a handful of American juniors.


• If a system assigns identical ratings to two players who play each other in a match, we credit that system with half (0.5) of a win and half (0.5) of an upset in the analysis.

Thoughts?

So there you have it. We at Tennis Recruiting are very proud of our ranking system, and we have tried to back up our claims with numbers.

If you have thoughts to share on our rankings or analysis, feel free to leave a comment or ask a question below. We will make every effort to answer any questions you have about this study in the comments section below.

Leave a Comment

More Ranking Articles

19-Sep-2017
Comparing Rating Algorithms
During the U.S. Open we posted information on social media about matchups involving American players - previewing matches by displaying rating and ranking differences and highlighting wins by underdogs. You might have noticed that the various rating systems often agreed on their favorites. But what about the close matches where they disagreed? Let's take a look ...

13-Jul-2016
Behind the Tennis Recruiting Rankings
This week marks the 569th consecutive week that Tennis Recruiting has put out graduation-based rankings of American junior boys and girls. Rankings are front and center on Tennis Recruiting, and people often ask us how our rankings work. Today we describe our ranking system ...

19-May-2016
What Is An Upset?
Tennis Recruiting is a website that rates and ranks junior tennis players. One of the questions we get most often from our users is, "What exactly is an upset?" There are many possible definitions of an upset - this article explores the question and puts forward an answer.

 

Ranking Article Index |

Subscribe