The Universal Appeal of Tennis Ratings

The appeal of ratings for tennis seems nearly universal. That is, once people understand how tennis ratings can be used to enable “Level-Based Play” and how tournaments organized around such ratings might positively impact players, they are almost universally inspired to learn more.

At the March 2018 meeting in Lausanne, Switzerland, the ITF Board of Directors approved the implementation of a new International Tennis Rating system.

Tennis Ratings are not new. There are numerous sanctioned rating systems in use today: the USTA’s NTRP, the ITF’s ITN, the LTA’s “British Tennis Rating”, France’s Systèmes de classement nationaux. In fact, there are so many rating systems that the ITF provides a conversion chart. ELO Ratings are also popular with researchers, statisticians and some commentators, and you can find live ELO Ratings for professional players online, while a variant of ELO is used by the Tennis Recruiting Network for juniors.

So why would the International Tennis Federation be pursuing yet another rating system? This is a question that can only really be answered by surveying some recent developments in what may be affectionately termed “the tennis universe” (rarely comic, sometimes darkly comic).

During the past year one rating system in particular has really entered the spotlight: The Universal Tennis Rating, or UTR. And it has people talking. After several years of gestation, UTR is gathering momentum. At the beginning of the year Universal Tennis announced a new CEO, along with a major round of funding; and a number of strategic partnerships were put into place, including Oracle, The Tennis Channel, and World Team Tennis. (Investors include executives from both Oracle and The Tennis Channel).

UTR differs from many other rating systems in that it is calculated based on match outcomes, including games won and the strength of opponents. In this regard it is more comparable to ELO Ratings, TRS (Todd’s Rating System), and Tennis Prestige Scores than to the more familiar rating systems. UTR also differs in the large variety of match data sources it includes in its calculations.

But the most significant difference is that the “UTR Powered by Oracle” system is promoted and backed by significant commercial interests.

I’ve long been a fan of UTR, and I think that over the past several years I could even be considered somewhat of an evangelist, though I’ve not written about it previously. An Open Source tennis tournament management system I’m developing exports match data in the UTR format; in 2017 I had the opportunity to submit a year of match data for a national federation; and for several years, while surveying the ranking systems used by many European countries, I’ve been testing the ability of UTR to predict match outcomes at Tennis Europe Junior Tournaments, and sharing the results with others.

It is no secret that many junior ranking systems are woefully inadequate. The boundaries between age categories can be particularly problematic: in some organizations it is possible for players to transition to a higher age category with inflated rankings which badly skew seedings; this is compounded when families (and coaches) with greater resources travel absurd distances at great expense in attempts to scoop points. The touted benefits of a ratings system such as UTR in this context are abundantly clear.

Disruptive Platform

Universal Tennis is confident that UTR will revolutionize tennis and “grow the game”; it showcases many ways in which UTR is beneficial. UTR has also been described as a “disruptive platform” (even on the Universal Tennis website). While I truly believe that ratings can be “good for tennis,” I also think it is important (and interesting) to explore how any disruption caused by the UTR platform might manifest itself in the tennis universe, and how “revolutionary” aspects of the Universal Tennis game plan may be related to some other announcements made by the ITF earlier this year.

Two recent events began to bring this issue into focus for me.

Working with a number of national tennis federations has given me a glimpse into the kinds of challenges federations face with respect to data management and their most basic IT systems, including their websites. This is the year of GDPR, General Data Protection Regulation, and one of the federations I’m familiar with was already experiencing some pushback on the scope of player data they were making available. Before any additional data could be shared with Universal Tennis there needed to be a legal framework in place to address privacy concerns.

The first event that got me thinking was reviewing the Universal Tennis “Data Sharing” agreement and realizing that it provided absolutely no benefit to the federation while granting “worldwide, non-exclusive, perpetual, irrevocable, transferable, sublicensable, royalty-free” rights to Universal Tennis, which would be able to “freely use, reproduce, display, distribute, modify and create derivative works” from all shared data. As much as I wanted to continue providing Universal Tennis with the data generated by my tournament management solution, I could not convince the federation that there was any real upside for their organization.

The second event was the opportunity to address a number of national federations at the 2018 Tennis Europe Annual General Meeting in Budapest, Hungary. I was invited to present my project, and began to reflect not only on how my offering might coexist with federations’ existing IT investments, but also on my professional experience co-founding a company which helps global organizations integrate both their internal applications and their B2B supply chains.

As I reflected, I had to ask myself: “How did I get here? How did my hobby visualizing tennis statistics turn into writing an Open Source tournament management platform?” The Answer: Frustration.

As both a tennis parent and a neophyte data visualizer I was frustrated by a lack of data, and the poor quality of data when there was data to be found (embedded in spreadsheets, CSVs and PDFs; full of errors). At the end of the day, it was all about the data.

The “fragmented and decentralized landscape” of tennis presents huge challenges to players, to families, to researchers, but also to federations. It occurred to me that “Data Silos” and vendor lock-in are afflicting tennis federations in the same way that they beset Fortune 500 companies (a battle I was fighting back in the 90s). Software vendors aren’t having it easy either.

The result of these musings, drawing on my background working with standards organizations such as the World Wide Web Consortium (W3C), was a proposal to create a set of software standards for tennis. At the AGM in Budapest I presented: “TOSS: Tennis Open Software Standards”.

“The development and promotion of TOSS would not only help set data standards, but also promote an environment where systems could provide better transparency, and where the creativity of university and private researchers and other software vendors could be unleashed.”

I didn’t intend to offend Universal Tennis, but some perspectives generated by the exercise of writing the TOSS document do point to ways in which UTR could be considered disruptive to existing stakeholders. This is no way means that I am opposed to Universal Tennis. On the contrary, I wish them great success. My analysis of some impacts their (as yet unannounced) business model may have should be taken as constructive criticism and not construed as adversarial.

Universal Tennis is, at present, primarily a processor of data produced by tennis tournaments, data which is made possible by tournament organizers, including federations. UTR, indeed any rating or ranking system, cannot exist without this data. UTR is at the center of everything that Universal Tennis hopes to achieve, which means that match data, and player data, are central to the long term success of Universal Tennis. So the question becomes, how might the aggregation of so much data about tennis, in a commercial entity, disrupt the world of tennis?

Gold, Oil, and Data

If there’s one thing we all should have learned by now, living as we do in the age of Google and Facebook, it’s that if you’re not paying for a service, then in some sense you are the product that is being sold. The gold that underwrites almost every free service is data. The Economist, CEO Today, and many high profile financiers and consultants have taken this position. At the same time, the misuse of data has registered a number of high-profile hits recently… with a firm at the center of one of the controversies even being forced to close up shop. Our relationship with data has become as full of international intrigue and global politics as our relationship with oil.

“One positive outcome of this spotlight on data is it will make everyone more aware and vigilant — governments, companies and us as individuals.”

I think that in the case of Universal Tennis we all need to be vigilant; while the universal adoption of one way of rating tennis players may be desirable, controlling a rating is not a path to profit… it’s a safe bet that the data that makes the generation of ratings possible will be the basis of the revenue generators that Universal Tennis eventually puts into place. Whether it attempts to become the Facebook of Tennis, the Amazon of Tennis, or the “one stop shopping, ecosystem of tennis” remains to be seen, because no business model has been declared, but these are the models that ought to be considered when trying to understand where things may be headed.

Universal Tennis should be applauded for its success in raising awareness of the benefits that can be derived from the use of rating systems. I believe it is what the founders intended, and how they originally conceived of doing something “good for tennis”. (I understand that at one point the founding team considered becoming a non-profit). The genesis of the project that has become Universal Tennis was passion, not profit, and while passion for tennis is clearly still a very strong motivator for some members of the team, UTR is now embedded in a for-profit entity, one that is well funded by silicon valley based venture capital, and can no longer afford to be purely “mission driven.” Financing can only carry a company for so long, and investors will eventually be looking for a return. The playbook being followed is a familiar one, with the land grab (data and visibility) being the first order of business.

No one knows how the data being provided to (or otherwise acquired by) Universal Tennis may eventually be used. A primary consideration for those that generate the data should be whether a future offering could conflict with or end up competing with the organization that provided the data in the first place. My guess would be that the Universal Tennis web properties will eventually be used for retail of goods related to tennis and advertising of products and services sold by third parties. With a big enough center of gravity that would definitely affect existing online retailers who focus on tennis, akin to Amazon entering a new market.

Online retail would not necessarily impact national tennis federations, but website visits and clicks taken away from their websites could decrease the enthusiasm of sponsors, on which many federations depend. We have to wonder whether results posted on federation websites will be marginalized by results posted elsewhere, and how tournaments organized based on UTR might coexist with tournaments organized based on rankings… who will do the organizing, and will up-to-date UTR ratings (and ratings histories) be allowed to be posted on federation websites if that takes away from traffic that might otherwise go to Universal Tennis?

What will the relationship between ratings and rankings be if players are chasing ratings? Would it be wise to base rankings on ratings if those ratings are non-transparent, based on a proprietary algorithm? And what happens if instead of the emergence of the Amazon or Facebook of tennis we witness the implosion of the Pets.com of tennis? What will happen to the proprietary algorithm used to calculate UTR?

A lot of questions to which I must add another: what is the exit strategy for the current investors, and what might the impact be of a transition from venture capital backed startup to wholly owned subsidiary or even publicly traded entity? Having been through this process several times myself, I know the pressures it can put on management teams and partnerships, and I wonder whether it would be “good for tennis” for a proprietary algorithm, in such circumstances, to be heavily relied upon by the existing institutions which have nurtured the development of the sport until now.

Proprietary to Protect

Why must the UTR algorithm be proprietary? Being familiar with other rating systems, I’m not convinced that the intellectual property behind UTR is where its value lies. To my knowledge, no comparative analyses have yet been published to suggest that UTR is superior, and no rigorous scholarly studies have been undertaken, or made publicly available, which demonstrate its predictive power (though some comparisons have been made).

UTR is valuable because of partnerships and good marketing, because of how it is used as a tool, not because of the way it is calculated.

When queried, Universal Tennis has reportedly suggested that if the algorithm for UTR were known then parents and players would take advantage of that knowledge to boost ratings; in other words, that UTR is proprietary, at least in part, to protect players from overzealous parents (or managers?), and that being proprietary can protect the integrity of the rating itself.

In fact, concerns have already been voiced that some players/families are attempting to game the system to boost UTR ratings (as they have long done in chasing rankings). No formal analysis has been done, to my knowledge, to understand the relationship between disparate pools of players, so it is not known to what extent it is possible to affect ratings by competing in loosely connected sub-groups of the player network, but it does seem plausible, and a case can be made that crossing boundaries between categories is a scenario where exploitable conditions might occur.

A perfect mixing of players would not be possible unless all players played UTR rated tournaments all of the time, with managed geospatial distributions … but that’s unlikely. So we may consider whether there are circumstances when ratings might be better calculated based on matches from only sanctioned events or only officially submitted tournament data. (The majority of data used to calculate UTR is scraped from websites, and there are duplications, particularly when player names do not exactly match. Player IDs are often unavailable in scraped data, and players frequently have multiple IDs from different systems).

It’s not just that UTR is a proprietary algorithm… Universal Tennis is the only entity which gets to decide what data gets fed into the algorithm and when… and at the moment they are accepting unsanctioned data from all quarters. Academy and high school sparring matches are fair game. There’s no data acquisition policy, or at least not a public one, so its not possible to tell whether ratings may be changing because of large inflows of historical data that were not previously included. This can lead to unpredictability, and now that players and US universities are relying on UTR ratings as part of the admissions process, the downstream effects of this lack of transparency can have real consequences. How data is acquired and fed into the proprietary algorithm is as big a concern as the proprietary nature of the algorithm itself.

If my assessment of UT’s probable business model above has any merit, then clearly the largest market for Universal Tennis is recreational… but the biggest boost in terms of credibility and marketing power comes from the professional and pre-professional pool of players. Do these players really benefit from the use of a rating that includes data from all types of match play with no audit-able criteria or quality controls? I don’t know.

“…having a collaborative model, instead of just one data provider and just one integrity monitor is optimal”

Who can answer such questions? Who should be the ‘integrity monitor’ for ratings in tennis? Should data be sourced and managed using a collaborative model, which can benefit all stakeholders, or should a for-profit company be in that position?

Data Redux

It turns out that Universal Tennis is also generating match data which is fed into UTR calculations: in addition to the non-sanctioned, UTR based tournaments which it organizes, it has been working with groups such as the Intercollegiate Tennis Association (ITA) to help rationalize how tournaments are structured. A natural offshoot of this aspect of their business is the development of their own Tournament Management System, which I understand will be offered for free to the global tennis community.

Perhaps the most substantial benefit of offering a tournament management platform is that it presents the opportunity to fully control a data source; no web scraping will be required, and the support burden of making corrections to match results can be significantly reduced. (It’s not clear, at present, what happens when match results are “corrected” on a site that is being scraped… does the incorrect data persist in the UTR system or is re-scraping periodically used to re-validate… are all fixes manual with changes made only when someone raises an issue? This is one of the many scenarios where a set of standards such as those contemplated by TOSS could be extremely useful in defining a data pipeline that could handle the propagation of changes to all stakeholders).

To some extent the development of their own platform will put Universal Tennis into competition with established vendors of tournament software. If the largest market opportunity for Universal Tennis truly is recreational players, however, then this shouldn’t be of immediate concern to providers of the most comprehensive solutions, such as Visual Reality, because large segments of the tennis universe (clubs, community centers, high schools and universities) are still using relatively primitive software, spreadsheets, or PDF printouts and aren’t looking to make significant IT investments.

Vendors of court management solutions might be alarmed, because this is a natural complement to tournament management, and it is an interface point where Universal Tennis could move much closer to the target audience.

If Universal Tennis’ tournament manager is to be used by many small-scale, non-federated organizers, how are players from such data sources to be matched with sanctioned results, and who can attest to the veracity of unsanctioned data coming from, e.g., independent clubs (all over the world?) What should be the mechanism for tracking players as they move between organizations, as they change clubs, cities, and even countries, and who should be allowed to submit data on their behalf?

Another consideration will be the rights that are to be granted in exchange for the use of a free service. Will match data be available to tournament organizers for use outside of the context in which it is created? Can this data be submitted to other ranking and rating platforms, if desired, and used for other projects which organizers may wish to pursue in the future? These are issues that need to be clarified upfront.

I made a reference to “vendor lock-in” above, and this term can usefully be explored here. Lock-in occurs when a customer becomes dependent on a vendor such that they cannot make changes without substantial switching costs. Recently I’ve been speaking with one entity that has been put in the position of scraping their own data off a web site that they are paying for — the vendor who built the website refuses to provide access to raw data for fear of losing lucrative monthly maintenance fees; other groups I’m familiar with are charged extra fees for API access to their data outside of the context of the application which creates the data.

Lock-in doesn’t always occur because of vendors’ product design goals; still, raising “barriers to entry” is a well established business strategy, and the existence of such barriers figure into the valuation of all technology start-ups.

Collision Course?

It is in the context of the ascendance of UTR and the critical importance of access to data and the warehousing and maintenance of data that I want to explore a recent decision by the ITF, in the midst of many other significant changes, to implement a new “International Rating System”.

To date, there has been no formal announcement of the ITF’s latest ratings initiative, and reference to it can only be found in the summary of the 2018 board meeting and in the job posting by the ITF’s Integrity and Development Department. Not much has been revealed about how this new rating, which has not formally been differentiated from the existing ITN, might be implemented or introduced, other than the statement that it can be used by all nations beginning in January 2019, and mention of a new amateur competition structure which will be linked to the rating.

At first glance it appears that the goals for the new ITN (or ITR?) do not significantly overlap with the aspirations that Universal Tennis has for UTR, but the timing of the ITF’s decision is interesting. Given my analysis above it would seem that the ITF is more suited to providing an “industry standard” for the long term, but it is clear that Universal Tennis is racing to define a de facto metric for all players, at all levels, worldwide, and that an ITF initiative could complicate this objective. Whatever the long term outcome, Universal Tennis should see the ITF’s announcement as validation.

Personally I believe that the context within which each rating is used will be a deciding factor in how this plays out for the long term… and by context I am referring both to the overarching data management strategies put into place as well as the competition structures which use each rating.

The ITF initiative is embedded in a broad development strategy that includes the introduction of a Transition Tour (which can help emerging players “stay local” and reduce the impact of extensive travel in the early stages of player development) as well as incentives for federations to invest in and improve digitalization; it does not appear, at the moment, to contemplate the recreational landscape, though it is clear that in earlier phases of the ITN the growth of the game at the recreational level was recognized as a key activity, and it’s probable that if the project is seen as successful the new rating will be made available to players at all levels.

Digitalization for many federations is a significant challenge. In addition to the structural hurdles when there is not sufficient funding, there is the fact that most tennis organizations don’t have in-house familiarity with many of the technologies required to maintain player and tournament databases; when they do have funding they are often at the mercy of 3rd party providers. The extent to which the ITF can offer resources and guidance to bolster such investments will be crucial to these federations’ participation in development activities which attempt to leverage ITF ratings.

The Universal Tennis ratings initiative is embedded in a for-profit venture that is seeking to include as much data generated by tournament match play as possible, and will improve digitalization by offering applications and services which may be paid for by advertising and consumer spending, which means that the largest pool of players, recreational, must be central to the strategy, though the use of such tools by organizations such as PTR, USPTA and the ITA could reach well into the pre-professional market, while the partnership with World Team Tennis includes professionals.

Universal Tennis is also seeking to partner directly with national federations. At the moment such partnerships are primarily data sharing agreements (as discussed above), but there is also the possibility that Universal Tennis is now or will soon be offering consulting services based on the application of UTR to national tournament structures, and there’s no reason their free tournament management system couldn’t be offered as well, though in many cases the (low to nil) cost of a product or service alone is not sufficient to address the pre-existing technical and organizational challenges.

Clearly the ITF doesn’t have the near term access to cash reserves that a recently funded startup can command, and Universal Tennis has a significant head start with respect to mind share and experience in the application of their rating, but if the ITF rating offers transparency (reproducibility) and embraces the large community of third parties interested in contributing to the refinement and broad application of an open standard, it may win the high ground with existing stakeholders, including the largest federations who are still responsible for the production of most of the relevant data.

I don’t think the ITF will be involved in scraping data from tournament results maintained by Universal Tennis, so unless the organizers who use the tools offered by Universal Tennis also submit data, indeed are enabled to submit data, to the ITF rating system, it won’t be included. However this shakes out, it appears that in the near term players will have multiple ratings (UTR, ITN) calculated using different formulas and different pools of tournament data in addition to the multiple rankings (regional, national, international) that they already have; and each metric will be used in different contexts.

It’s possible that Universal Tennis could take an open stance, participate in and even lead the creation of a set of standards related to player metrics and the data flows that underpin them; it’s what I’d really like to see… but I’m not holding my breath. The ITF’s “International Tennis Rating” may put them on a collision course with Universal Tennis, and it is probable, but not inevitable, that we may soon witness conflicts as different organizers struggle to determine the path they will follow with respect to ratings… whether any such confrontations in the “tennis universe” will escalate into a “civil war” or an “infinity war” remains to be seen.

Good for Tennis — An Appeal

I believe there are a lot of people who want to do good for tennis; I know there are many, many people who love the game; there are also those for whom tennis is a means to an end — and all of three of these positions can be true for a single individual, and represented to varying degrees within the diverse organizations that participate in the “tenniverse”. This is as true of the ITF as it is of Universal Tennis and each and every federation and vendor of software connected to the game, and it can lead to quite a bit of disfunction.

I’m a nobody in the world of tennis; I didn’t start playing tennis until my 40s, neither of my parents played tennis (one grandfather was a college star and played through his 80s); but like many other tennis nobodies, I’m a parent of young players who are just starting a courtship with tennis that may turn into a lifelong relationship. I believe that if our approach to tennis is healthy, then by the time they are ready to head off for college the things they’ve already learned through tennis will be more valuable than what comes after. So I’m focused on the family experience at the earliest stages of player development, and I think it’s important that all stakeholders remember that this is the origin of the tennis universe, the activity that makes the bang possible for everyone trying to make a buck. The reason it is important to remember is that it is a confusing time… it’s not easy to chart a course through unfamiliar territory and make sense of the healthy and unhealthy relationships between all the competing interests.

What’s good for tennis may be defined in many different ways. To some it is simply the growth of the sport, which may be measured by the number of registered players, the volume of goods sold, the number of jobs created and sustained, the size of the television audience, or the number of professional competitors who can actually make a living; for others it is the quality of the experience, which can be measured in terms of player and family satisfaction with coaches and clubs and tournaments and may somehow be linked with the number of competitive matches which are possible, but probably has a lot more to do with the depth and longevity of the relationships that develop and whether people believe the experience has been a worthwhile investment of both time and money, after the players step off the court. All of these considerations are no doubt connected. It is simply a matter of where we put our focus and attention when making assessments.

Yet another perspective is that more technology is good for tennis. Better statistics, instant video analysis… more data. Truth be told, and despite my successful career in software, I’m a little skeptical of the “goods” that data and technology can deliver, but I know very clearly the harm that can be done when data is mismanaged, when promises are made that can’t be delivered, and when there are no competitors — when products are developed in a vacuum or monopolize a market. Healthy competition is a requirement if we wish to raise our game to a higher level, in business as in tennis. And so my appeal is that those who pursue the application of technology to tennis compete on quality, not the quantity of data that can be amassed and controlled. And that means paying a lot more attention up front to how the data is structured (standards) and how/whether it can flow to the places where it can do the most good; it means being aware, up front, of the data rights that are granted or denied in every contractual relationship.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store