Will we ever get to a point we can use analytics to judge players relatively accurately?

[Smoothbutta]

At this point in the databall era, can we use some combination of VORP, WS/48, and RAPM/PIPM or some other metrics to evaluate players not perfectly but to some reasonable extent? If not now, will we be able to at some point in the next decade?

Also why are there no great sources for RAPM or PIPM stats that are both updated and go back to 1996? Please share if they are somewhere reliable.

[Dutchball97]

I don't think we'll ever get to a point where we have analytics that paint a 100% accurate picture but I do believe that using an aggregate of many different stats can already evaluate players to a reasonable extent. Personally I use composite stats like EPM and LEBRON the most and consult boxscore stats (BPM and WS), +- and on-off to look at the more isolated parts.

[Cavsfansince84]

I think we got to that point 5-10 years ago tbh. It's just that you have to use a combination of them and some of the better ones are likely kept under wraps by teams and data only goes so far back. It's subjective though because any data people don't think meets their own idea of how good a player is tends to be rejected. That's human nature when it comes to which analytics people tend to use more. How do we know when we've found the perfect advanced metric? Do bells go off in the heavens or something? There's no way of knowing.

[Smoothbutta]

I think we got to that point 5-10 years ago tbh. It's just that you have to use a combination of them and some of the better ones are likely kept under wraps by teams and data only goes so far back. It's subjective though because any data people don't think meets their own idea of how good a player is tends to be rejected. That's human nature when it comes to which analytics people tend to use more. How do we know when we've found the perfect advanced metric? Do bells go off in the heavens or something? There's no way of knowing.

Yea but like, where is that data then? Is there any website that is still active that gives data like EPM/RAPM/LEBRON since 1996 and is available for us to use?

[penbeast0]

What do we mean by "relatively accurately?" If you mean better than by people's individual eye test, we reached that point on offense a long long time ago, though not on defense. Tracking data and the like make our defensive evaluations a lot better too. If you are looking at factors like locker room compatability, heart, ability to rise in the clutch, and even BBIQ, we have a tough time even figuring out what we want to measure before we even get to the issue of can we measure it. If you mean "perfectly," it will never happen.

[OhayoKD]

At this point in the databall era, can we use some combination of VORP, WS/48, and RAPM/PIPM or some other metrics to evaluate players not perfectly but to some reasonable extent? If not now, will we be able to at some point in the next decade?

Also why are there no great sources for RAPM or PIPM stats that are both updated and go back to 1996? Please share if they are somewhere reliable.

Depends on what specifically you're looking at and your criteria.

Will just link my approach to player eval(maybe a little out of date now, but includes all the various stats there):

https://forums.realgm.com/boards/viewtopic.php?f=64&t=2248282&start=300

Spoiler:

Well alright, I suppose I could summarize my general approach. I generally like to start as broad as possible and then narrow down. General goal is to get a general range of value for an individual player at various points, then adjust for context, andthen, if era translation is required, apply the precepts of "league generally gets better" and "scarcity is value" and map to specific strengths and weaknesses.

For now I'll focus in on my era-relative placement process.

Step 1 is to just accumulate holistic evidence. What is the WOWY(always start with the biggest possible samples imo), what are potential sources of team improvement or decline, what's the regularized stuff saying(ideally look at volume and per-possession effiency), on/off, what is box-stuff saying, rs and playoffs, key to map out as much as possible as opposed to simply choosing a year based on perception. I like to look at what the high view, and the low views are. Even with a specific sample of WOWY, you can get different extrapolations based on different decisions(do you use srs or record? do you treat a player as a like for like replacement, if there's a minuites restriction, do you adjust?) In the 97 thread, that was actually a big focus of the mj discourse. Also keep in mind starting points, generally easier to elevate a 20 win team by 20 than a 50 win team(though thats not neccesarily a hard-set rule depending on the player type, some truth to "cieling raising/floor raising distinction"). With limited available, "weaker signals" may be useful to look at(olympic point diff record/pre-nba dominance with russell and kareem, partial rapm form peak mj) as a supplement.

Step 2 is adjusting for context, analyze potential sources of improvement, analyze spots where player may grow in value or decline in value, maybe establish predictions of where they should and shouldn't output value and then see if we have test-cases(so for Lebron you have the theory of cieling raising/spacing dependency and then you have spots where he defies it to some extent(2015, 2020, 2012)). Consider situation, is team having coke crisis, is FO antagonizing player unprompted, is coach competent(in this light Hakeem hitting some of the best notes of his era is very impressive(best examples of lift arguably, most impressive win maybe(lakers), single-star title, ect)). Also consider if individual metrics rising or dropping correspond with team rising and dropping(you can do this with defense, offense, or holistically).

Step 3 is weighting the holistics. Again sample size is a big consideration, but also specific player make-up. If there's outlier WOWY, then artificially capped rapm (and its derivatives) will probably be misattributing value, off-ball creation or paint protection as strengths will probably lead to weak box-score rep. Helio's may be better at elevating from lower points than they are from higher points(lebron/magic) and reverse may be true with non-helio's(curry/jordan). WOWY always has some utility as giving teams the opportunity to adapt, and going off a bigger sample is a strength vs more modern impact analysis which is looking at lineup data(as it should). To me it's like a sniff test, and if certain things are consistently disagreeing with it while others aren't, I get skeptical.

Also important to remember the time-based limitations of data. Kareem's RS ws/48 from 71-73 looks goated(and tracks with what a WOWY+context analysis of 71-75 would lead to imo), but his playoff score looks horrible because...the data is incomplete. Whatever you think of RAPTOR(apparently it ranges from sub-pipm to only behind direct-rapm depending on the test), if you're using it to assess older-era players, its basically stripped down to a PER-esque metric as player-tracking/plus-minus(which in every test seems to make data more predictive/stable) are gone. D-PIPM can do a bit better because its box component is tied to d-rapm but the offensive accuracy plummets(box-component is sub raptor's while full stat is more predictive and taken more seriously by nba teams).

Obviously consider sample size(wowyr suggests Russell is winning 11 rings with 35 win help, but that's only off 2.2 games a season, 82 game sample from 70 and 57 are probably better too work off), and consider additions/subtractions and the effect(Lebron is still anchoring an elite defense without second best defender in 09/10, Kareem is leading 62 win pace team without Oscar, Rodman looks really good in impact stuff(kd-esque wowy), Hondo sees their production skyrocket in 1970 and replacement for bill is drafted, defense rises and collapses with Oakley, ect, ect.)

Step 4 is to look for replication, if a player scores at or within range of the top at basically everything multiple times over in multiple contexts(Kareem and Lebron more or less) in close to every possible frame(playoff, rs, playoff+rs, floor-raising, cieling-raising, blah blah blah), then my instinct is to trust consensus and rate them accordingly(which is why right now, Lebron and Kareem are my two best post-russell/wilt peaks. Lebron actually does the best in terms of replication imo (by a margin), but Kareem has the "led a goat rs and po level team with probably not spectacular help" feather in his cap). Also accept uncertainty(we don't have everything on Russell, but what we have indicates he has a GOATED era-relative prime, and Wilt can scale off that to an extent. No reason to think in black and white and dismiss all that due to "not being enough info".)

Step 5 sort of builds for step 4, but basically its to look for resiliency, playoff performance, whether performance drops the longer a series goes on, whether they can remain impactful when certain parts of their game are hindered(2015 lebron and 2019 Giannis are good examples of this. Curry outplaying KD while injured, Flu game, also good examples.).

Step 6, assess off-court stuff. Is a player causing problems/instigating, are they operating as secondary coaches, weigh the good vs the bad, and remember that just because a player happens to win in a specific context does not mean that what they're doing off the court is positive(better to look at general trends to determine what is good or bad imo). Bill Russell is the clear GOAT here

Finally Step 7 is to consider longevity/sustained excellence. Even if you only care about peaks/primes, it's a good idea to remember that players who play longer will generally see averages dip and have more "bad" moments. If you don't want to credit players for that, fine. But don't penalize them for it.

With all that considered, I'm going to offer my own peak/prime/career val in case you're wondering how this can shape out. You are welcome to scrutinize/challenge anything here. Keep in mind this is purely era-relative and post shot-clock.


Prime
1.Russell
(Gap)
2. Lebron
3. Kareem
4. Wilt
(Gap)
5. Jordan
6. Hakeem
7. Duncan
8. Magic
(Gap)
9. Bird
T-10. KG/Curry

Peak
1. Russell
(gap)
2. Wilt/Lebron(1 year lens produces outliers of russell-level rs and playoff value arguably)
(gap)
4. Kareem
(gap)
5. Duncan
T-6. Jordan/Hakeem
T-7. Shaq/KG(1-year would be at t-6 or t-5 FWIW)
T-9. Bird/Magic
T-11. Giannis/Curry

Career Val.
T-1. Lebron/Kareem(i consider how good players were pre-nba relative to the nba)
2. Russell
(Gap)
3. Wilt
T-4. Jordan/Duncan/Hakeem
T-7. Shaq/KG
(Gap)
9. Kobe
10. Magic
T-11. Dirk/Bird

Tell me what you think!

Some additional contextual considerations(and time machnine/direct player comparisons:

Spoiler:

[spoiler]

At the buzzer but...

Will expand on the rationale later but I'll be voting for POY and leave the rest to my betters. Was thinking of trying EOY, but with the "double-dipping" considerations, I'd probably just confuse things.

Criteria/Methodology for player evaluation
For now, player-assessments are strictly era-relative. Am considering a shift to "impact averaged over time" or modernist-era-translation as factors, but for now it's really just how much you increase a random team's chances of winning it all. I'm not exclusively looking year to year necessarily(other teams may take a season or post-season to adjust), but it's not nothing. Surrounding years also matter to me, and for a rough rant explaining the steps, you can look here(I think I've fine-tuned that in ways but still):
https:/Wit /forums.realgm.com/boards/viewtopic.php?p=103144819#p103144819

Other Considerations
Basing these guesses on historical and contemporary results, I'm working on a few assumptions:

-> Protecting the Rim is the most consistent source of value across different situations and contexts:

Perhaps at a higher theoretical treshold the gap between playmaker stacking and wing-scorer stacking becomes evident, but that treshold hasn't been reached and "paint-protectors" currently look like the least "situation-dependent" archetype. At an individual level, Duncan and Russell are probably the quintessential metronomes if we go by team success and if we go by individual impact, Russell and Kareem stand-out in terms of a lack of fluctuation. From rookie year to 1980 Kareem's "impact" stays pretty consistent with small postseason samples being where most of the fluctuation happens. Russell is still winning with seemingly average help right as he's about to retire.

-> Offensive players who are highly efficient-creators(think passer-rating instead of box-creation or ast:tov%) are the most resilient in playoff-settings
-> Players who function as on-court coaches(telling teammates where to go, prompting coaching decisions at key spots) on one or both-ends are extremely resilient to changes in situation and provide value higher than their physical production might indicate
-> Players who cannot require more specific conditions to reach their situational ceilings and therefore are curved down

it's not just about what you create. It's also about the quality of what you're creating AND how much you're leaving on the table with suboptimal decisions. Players on this tier have better discernable offensive "lift" than players the tier below, and often this is blamed entirely or pre-dominantly on "this is just because of who their teammates are", but I actually think the real source of this offensive advantage is the "quality" of what they're creating(and some of the backseat coaching stuff has an off-court effect that can't be tracked via impact stuff):



Don't have access to the numbers(paywall) rn but passer-rating also sees this. Curry and Jordan graded out as comparable or right behind creators in a pure volume metric like playval(based on ben's bpm which is using assist totals I think) or Box-OC, to guys like say Lebron, Magic, and Nash, but they had teammates telling players where to go(draymond/pippen respectively), and don't make the best possible reads as often(I think ben said it was something like 60% vs 80% of the high quality passes in his peaks video and we have the "good passes" number above).

Incidentally they don't seem to have the same level of offensive lift in the absence of a specific structure where those decisions are delegated to someone else:


By comparison, the best pre-triangle Jordan stretch(with Jordan arguably at his peak) sees a 52(Ben) or 53-win(E-balla) team over a 30-game sample going at +4.4 offensively(you can reach a +4.6 if you swap minuite distributions for the 5th and 7th mpg guys for 20 games and ignore the team didn't actually improve), Curry wasn't close to leading all-time offenses(and had worse metrics than both westbrook and durant) with Draymond on the bench.

-> All else being equal, being able to impact without the ball offers an advantage of being reliant on having the ball(KD vs Westbrook):

However, Nash’s situational value clearly changed from Dallas to Phoenix, as multiple APM methodologies demonstrate marginal impact in Dallas and seismic correlations in Phoenix. Improved health and the freedom-of-movement rule change were both factors, but I view these competing measurements as a classic case of fit. Similar to LeBron and Wade, Nash’s style of play created some diminishing returns. Unlike LeBron or Wade, Nash’s unheralded background and diminutive stature masked his poor fit in Dallas. Nash was more of a situational floor-raiser who could wash out in certain lineups next to ball-dominant scorers; he wasn’t as versatile as someone like LeBron, so pairing him with other centerpieces didn’t automatically supercharge such teams.

-> Players who disproportionately rely on being elite man-defenders to boost defenses are less likely to reach their situational ceilings than players who are not
-> Players who have limitations as ball-handlers are less likely to reach their situational ceilings than players who are not(There are levels to this, a Kareem or Jordan is dramatically less reliant on lower-ball-handling load to maximize their scoring value compared to players like Bird or Durant)
-> Credit is warranted when a player succeeds in-spite of roster-turnover or fo-drama, as it is harder to perform well in such situations
-> Failure is more forgivable in such circumstances(If the Suns go ballistic with an off-season to figure things out I'll retroactively raise Durant's 2023 a bit)

Off-court "impact on winning" matters, but I am going to be going off direct accounts and tangible actions/examples as opposed to media-narratives(am going to get into this with certain "coach-killer" claims at some point). Steph may be considered the leader of the Warriors, but as Dray is the one who is telling teammates what to do when he's on the sideline(you could hear this on mic'd up in game 2), is called the emotional leader by teammates, and is coaching up young-talent, he gets the lion's share of off-court credit. Not too relevant to a single-season ranking, but I'm planning to copy and paste this preamble(mostly) for the top 100 so humor me.

Off-court "gming" good and bad, taking pay-cuts, ensuring teammates sign contracts, side-line(or on-court) coaching(and the actual results), all come into play. I am not going to assume "the best player" is the "leader" even though people(teammates included) tend to assume that, but I will account for what specifically players in the organization say a player did or didn't do.

What is "Likely" to happen or how replicable something is(across different situations) is also relevant(not as much in a descriptive POY ranking). This is an important thing to consider with injuries(not so much with POY voting but still).

Competition quality matters but in terms of winning championships, the top-end is more relevant than the bottom-end. Beating the better "best" opponent matters more than beating a better "2nd best" opponent. Can be applied to looking at how strong a "year" is or how strong the path to a championship is. Regular season srs is also less relevant than something like say "San's psrs" in comparisons featuring teams from the last 10 years.

With all that said

POY BALLOT

1. Jokic
He was the best regular season player. He also completed a three-year stretch where had a better argument for being the best player every single one of those seasons than nearly anyone(and yes that includes any 3-year stretch for Jordan or Lebron). The only players in history I think may have had a comparably good argument are Bird, Kareem, and Russell. He destroyed 1-year impact-witnqlw like wowy, lineup-splits, extended wowy, ect(note that RAPM and APM-derivatives are less useful differentiating between 1-year peaks), and those sorts of numbers remain nigh-unprecedented when you make adjustments like taking out Jamal. Taken at face-value that 1-year rs portfolio looks better than anything that's available for the likes of Magic, Bird, or Jordan and right up-there with the likes of 2016 Steph, non-09/10 Lebron, and 03 Duncan. He also played significantly more than either of the other MVP-candidates and coasted to end the season because the Nuggets secured the 1-seed extremely early. Note that his best-teammate looks pretty pedestrian over a substantial when we flip things.

In the postseason the Nuggets went 16-4 posting point-differentials of +8.8, +8.9, +6, and +8.1 with their worst performance coming against a Lakers side that decisively clamped the champs. If you were to use the rs/off as I tend to do(sample-size) and attribute the playoff-improvement to Jokic(more on that later), there are a handful of players who've posted years that look comparable. I do not think he is especially advantaged in terms of fit(I consider the 2019 Raptors and Kawhi, Draymond/Steph, and Pippen/Jordan, Lebron/Bubble AD more exceptional in that regard) and even with weaker competition(like pretty much every team that was similarly dominant), the Nuggets were properly great.

Jokic is also a goat-tier offensive player with the only players i'd entertain as more valauble era-relative on that end being Oscar, Magic, Lebron, Nash, and Mikan. He is a truly do-it-all-scorer(someone like Durant can see their scoring suffer greatly when they have to function as a secondary ball-handler) who is good from pretty much every spot on the court on good to great volume. He is an all-time tough-shot-maker, an all-time post scorer, an all-time mid-range talent, an elite inside-guy, and might just be the best pure passing talent in league history. He also is one of the smartest players ever and like a Lebron, Nash, or Magic has impact that goes beyond what you can physically see him do(and thus should get some of the credit for his teammates improvement). His only weakness here is ball-handling(and it's not so easy to "push-pace"), but for a center he is absolutely exceptional there(and a matchup nightmare even for an all-time versatile defensive big like davis).

Jokic's defense held-up alot better in the playoffs than I thought and though he faced a fairly favorable set of positional matchups, it's notable his defense was outright good in the finals and against a certain Anthony Davis(albeit an injured one).

No one was as good and I would hope he would go down as a unanimous #1 here. He should have gone down as the first B2B2B MVP in nearly 30-years.

Caveats His playoff "impact" is actually pretty pedestrian if you use the postseason for off and while I'm not too big on one-offs, this is the third-year in a row this has been true. He actually was on pace to post a negative on/off for the 3rd year in a row before the final game.

He also looks alot worse using larger samples. 15-17 Steph and 15-17 Lebron look alot better in the RAPM sets, and Embid also cooks him in both the 3-year and career stuff I've seen. His 3-year rs impact still looks good relative to the likes of MJ and Shaq(especially shaq) but he goes from a step removed from the very best looking seasons to a couple steps even using raw-stuff. Luck-adjustment flips things but on a 3-year sample using luck-adjustment feels wrong. I also think Murray and Gordon both had bigger roles in the Nuggets postseason improvement, and he ran into two teams affected with injury(Lakers and Miami) with the most favorable home-court in the league. It's also worth noting that the Lakers series looks lot closer-looking if you account for how high-scoring the games were(a 6-point mov in a 120-orating series is not the same as a 6-point mov in a 90-orating series). While statistically the Nuggets are an all-time team it's hard to say they'd be favored vs the 2020 Lakers who merely rank 40th if we use something like Sansterre's methodology.

All of which is to say, while I certainly have no issue with someone arguing for Jokic to be higher, as things currently stand, I am not viewing Jokic's 2023 as a "goat-tier" year. I would also favor "non-goat" years like Duncan's 03 and Hakeem's 93/94 if we're just talking era-relative goodness. Nonetheless, he's an obvious #1 in the most talented era of league history. Give him all the props.

2. Butler
I think he was an arguable top-10 player in the regular season. I think he was also arguably the best player in the playoffs until injury and even with injury he outplayed another top 5 poy guy to make the finals with a short-handed team. I think purely on level of play he might be lower, but I also think Butler deserves some credit for the culture that has seen undrafted talent blossom in to contention-calibre starting pieces. He faded in the finals, but in a year where most of his competition got neutered by injury, an okay, not great finals performance seems forgivable.

3. Tatum
This is quite simple. Tatum was probably the 4th best player in the regular season(apm-derivatives would support that) on the second best team in the regular season. He was also arguably top-5 in the playoffs. I do think Butler outplayed him h2h(consider competition) so he's 3 and not 2 but he is a do-it-all perimeter player who was one of the best in the league wire to wire on a team that suffered a fair deal of drama and nearly completed the first 3-0 come-back in history.

4. Giannis
He was the 3rd best player on the best rs team by record(top 5 by srs) despite his best teammate missing time. His injury is also less of a factor to me than say an Embid, because he was able to recover fairly quickly and had the Bucks not ran into an eventual finalist in the 1st round there's a good chance the Bucks suffer his diminished influence playing hobbled till he found health. You can criticize his ft-shooting but even if that career-worst shooting performance was not injury-related, there's a decent chance the Bucks advance anyway if Giannis's defense isn't severely limited. I put him lower than Tatum due to his injury but he's earned some benefit of the doubt so he stays on the ballot.

5. Davis
He was the best defensive player in the playoffs, a dpoy(per-possession at least) lvl player in the regular season, and performed an all-time defensive feat for a big somehow nuetralizing the Warriors P n R. As Aneigma has noted, his impact stuff is juiced based on a lack of backups(and is flatly worse than Lebron anyway considering the RS), but the playoffs matter most and he was clearly the Lakers best player come-winning time. I also don't like him being removed from the picture as factor in the Lakers making the playoffs as the Lebron-less Lakers playing like a good team post-trade on davis going ballistic was a crucial factor in them having a conference run to speak of. If Lebron was healthy or AD didn't step up his defense, I would easily take Lebron base thanks to alot of the "off-court" stuff, but as the playoffs matter more to me, AD makes the top 5 with Lebron a hair behind.

If he had played the way he did in the 1st 2 rounds against the Nuggets, he would probably be in consideration for 2 or 3.


HM:
'

1, Lebron - Clearly top 5 for me during the regular season when you consider context(is arguably top 5 if you just go by impact stuff noisy as they are for 1-year), with him looking very impactful on a good team and a bad team. An easy top-10 during the postseason despite tearing his tendon and comfortably the best offensive player and 2nd best defender on what, by point-differential at least, was the best challenger to Denver(I am still giving Miami the mantle of 2nd best playoff team everything considered). Anenigma made a great case for Lebron being on the ballot but it would be inconsistent for me to put him in the top 5 even though its tempting. I don't worry about "accomplishment". Just what a player offers in terms of championship likelihood. This year should serve as a big-wake up call for people who were treating him as a significant off-court negative or a guy who is limited in terms of his ability to adapt on higher-end rosters. He operated as a tertiary ball-handler and a small-ball 5 or 4 stretches and was still extremely valuable with the Lakers playing like a 69-win team post-trade(record, not sure what the net is) wirh him on the court. He did really well in lineups with the likes of Westbrook and really well with abysmal spacing. Being one of the smartest players ever is probably worth more than many think.

That he was willing to be deployed the way he was offensively and defensively should prompt some serious questioning for those pushing him as an uncoachable egomaniac. Ham probably doesn't do as well as he did with your traditional(in terms of temperament) all-time great.

2. Embid. He was the 2nd best regular season player and if he was healthy I think he'd be top 3. Alas, he was injured, again, and despite being one of the most talented players ever, his career turning a warning shot for large players hoping to stay healthy in the current iteration of the league.

3. Booker. If he didn't get hurt in the last 2 games of the suns season, I'd consider him for 5 or 4. He was excellent in the postseason putting up great scoring with good defense and underrated playmaking(the idea that Durant was some sort of Curry-like fulcrum is absolutely absurd) and ball-handling. He was also a top 10 rs player when healthy but alas he missed a large part of the seaosn. He also played pretty weak defensive competition and a really weak 1st round opponent. The Suns did take the most games from the nuggets but they were also outscored by the most. I would also expect they would have lost against any of the other contenders given health.

4. Curry. Great playoffs, but he was not top 5 in the regular season and he was arguably worse in the playoffs than a guy who didn't make my ballot despite being up a tendon and down 30k minutes of milage. Warriors lost with home-court to a non-finalist after being close to .500 in the regular season in a pretty good situation in terms of fit. Steph was also a clear 3rd best in the biggest game of the season and he was wearing down quicker than his aforementioned tendon-less nemesis.

I see no good arguments for him being top 5 honestly, but he's probably better than anyone I haven't mentioned yet. As I've mentioned before he does a lot better projected across time than he does era-relative.

5. Luka. A better rs player than a couple of the guys on my ballot but...no playoffs no party

[/spoiler]


Stat Sources:
-> Lineup-ratings -> PBP

-> WOWY/indirect/raw singlas -> Statmuse, Backpicks

-> RAPM (there are various, but the one that seems to be the most transparent is amhed cheema's)

-> Hybrids (you can find them on their given website)

-> On/off -> BBR

-> Box-stats -> depends on the stat, nba.com, bbr, synergy, bball-index, and of course there's sometimes value in creating your own while tracking film (lots of tracking has already been done here by other posters)

Generally I'd say "advanced stats" like LEBRON or whatever should be the last steps in most comparisons.

[Smoothbutta]

Stat Sources:
-> Lineup-ratings -> PBP

-> WOWY/indirect/raw singlas -> Statmuse, Backpicks

-> RAPM (there are various, but the one that seems to be the most transparent is amhed cheema's)

-> Hybrids (you can find them on their given website)

-> On/off -> BBR

-> Box-stats -> depends on the stat, nba.com, bbr, synergy, bball-index, and of course there's sometimes value in creating your own while tracking film (lots of tracking has already been done here by other posters)

Generally I'd say "advanced stats" like LEBRON or whatever should be the last steps in most comparisons.

Do you have a link to WOWY data on statmuse?

Do you have a link to Ahmed Cheema's RAPM data? I see some public charts that go to 2021 but nothing past that.

I never noticed the on/off tab on BBR, thank you

What are hybrids?

[capfan33]

Think its very difficult without stats based on player tracking and some sort of input for schemes and role. Too many variables compared to something like baseball, especially defensively.

[rk2023]

I second my guy capfan33 here

[LukaTheGOAT]

We can.

Understanding these metrics without watching the game would give you better ability to estimate player impact than 99% of NBA fandom.

[OhayoKD]

Think its very difficult without stats based on player tracking and some sort of input for schemes and role. Too many variables compared to something like baseball, especially defensively.

Depends on your bar for "accurate"

[OhayoKD]

Stat Sources:
-> Lineup-ratings -> PBP

-> WOWY/indirect/raw singlas -> Statmuse, Backpicks

-> RAPM (there are various, but the one that seems to be the most transparent is amhed cheema's)

-> Hybrids (you can find them on their given website)

-> On/off -> BBR

-> Box-stats -> depends on the stat, nba.com, bbr, synergy, bball-index, and of course there's sometimes value in creating your own while tracking film (lots of tracking has already been done here by other posters)

Generally I'd say "advanced stats" like LEBRON or whatever should be the last steps in most comparisons.

Do you have a link to WOWY data on statmuse?

Do you have a link to Ahmed Cheema's RAPM data? I see some public charts that go to 2021 but nothing past that.

I never noticed the on/off tab on BBR, thank you

What are hybrids?

You go to statmuse.com and enter queries like "2023 hawks net rating with and without trae young"

Some WOWY has been compiled already actually:
https://forums.realgm.com/boards/viewtopic.php?t=2353834

https://forums.realgm.com/boards/viewtopic.php?t=2310915

Ben taylor's backpicks top 40 usually features health-adjusted srs for concentrated stretches

You can also go to bbr and check season to season srs swings

Here is a breakdown of cheema's rapm with links to the original data and explanation for methodology:
viewtopic.php?t=2301003

The tab for on/off in bbr is "play by play"

and here's a 5-year variant:
https://www.thespax.com/nba/quantifying-the-nbas-greatest-five-year-peaks-since-1997/

[Smoothbutta]

Yea unfortunately Cheema's RAPM data only goes till 2021. So I still haven't seen any complete data-set of RAPM from 1996 to 2024 unfortunately, I'm not complaining btw just trying to understand what is and what isn't available.

I started compiling some players single season peak On-Off and career On-Off from BBR and it's interesting that Draymond's from 2016 is higher than every single other 2000s and onward player in the top 50 or 60 all-time. I understand sample size and taking one data-set with a grain of salt but that is still really interesting.

[OhayoKD]

Yea unfortunately Cheema's RAPM data only goes till 2021. So I still haven't seen any complete data-set of RAPM from 1996 to 2024 unfortunately, I'm not complaining btw just trying to understand what is and what isn't available.

I started compiling some players single season peak On-Off and career On-Off from BBR and it's interesting that Draymond's from 2016 is higher than every single other 2000s and onward player in the top 50 or 60 all-time. I understand sample size and taking one data-set with a grain of salt but that is still really interesting.

some things to keep in mind with on/off and it's derivatives(this includes rapm)

-> lineup effects:

a. co-linearity, when the stars of a team play heavy minutes together, on/off tends to go higher. RAPM can mitigate this but it's still just an approximation (curry and draymond, jordan and pippen, 2024 jokic)

b. staggering, when the stars of a team play substantial minutes separate from each other, the on/off tends to go lower as the off is inflated and the on is suppressed (ex: 2024 Curry with cp3, luka until this trade deadline, Lebron in Miami with Wade)

c. minutes load, players who play much higher minutes than everyone else on their team tend to see their numbers suppressed as they play with the backups too (ex: Duncan). Opposite effect with low minute loads (drob)

These can affect all stats, but on/off, largely based on spot minutes without a player, is super sensitive to this. WOWY minimizes this sort of bias but is relatively variable. RAPM is stable but approximates and can't be used year to year in the same way. Nonetheless, checking on/off against both when possible is a good idea.

It's also a good move to check lineup ratings(pbp) and/or rotation sheets

[Doctor MJ]

At this point in the databall era, can we use some combination of VORP, WS/48, and RAPM/PIPM or some other metrics to evaluate players not perfectly but to some reasonable extent? If not now, will we be able to at some point in the next decade?

Also why are there no great sources for RAPM or PIPM stats that are both updated and go back to 1996? Please share if they are somewhere reliable.

Well, I would argue that the fact that we call this the "databall" era is due to NBA teams using analytics to judge players relatively accurately.

So then it's really a question of what the specific threshold on your mind is.

For me, the gold standard would be a stat that is based on feature-extracting skills based on player tracking data. I feel like it's only a matter of time before NBA teams are doing this for particular skills (some will be easier than others), if they're not doing it already, but I'm not what along these lines will be available to the public.

In terms of the lack of reliable stats here, it's 2 issues:

1. People who publish these sort of stats tend to get hired to do similar work, often by NBA teams, and then they at best leave the site static, but sometimes take down the site.

2. There are no objective standards for these stats. When you make a RAPM, there are decisions you have to make with no clear right answers, and so when different people run RAPM, they get different results. This has a number of negative consequences for wide spread adoption as you might imagine. In practice what this means is that when a new source of RAPM comes out, the community evaluates what they see with a skeptical eye. If they see some wacko results they won't allege fraud or anything like that, but also won't tend to keep using that source.

[OhayoKD]

At this point in the databall era, can we use some combination of VORP, WS/48, and RAPM/PIPM or some other metrics to evaluate players not perfectly but to some reasonable extent? If not now, will we be able to at some point in the next decade?

Also why are there no great sources for RAPM or PIPM stats that are both updated and go back to 1996? Please share if they are somewhere reliable.

2. There are no objective standards for these stats. When you make a RAPM, there are decisions you have to make with no clear right answers, and so when different people run RAPM, they get different results. This has a number of negative consequences for wide spread adoption as you might imagine. In practice what this means is that when a new source of RAPM comes out, the community evaluates what they see with a skeptical eye. If they see some wacko results they won't allege fraud or anything like that, but also won't tend to keep using that source.

It's lightly damning that cheema is the only rapm source(setting aside scaled one-year from cryptbeam) that is transparent on methodology and formulas

[andyhop]

For me, the gold standard would be a stat that is based on feature-extracting skills based on player tracking data. I feel like it's only a matter of time before NBA teams are doing this for particular skills (some will be easier than others), if they're not doing it already, but I'm not what along these lines will be available to the public.

I wonder if anyone is modelling basketball like Liverpool model football.

Instead, he spent months building a model that calculates the chance each team had of scoring a goal before any given action – a pass, a missed shot, a slide tackle – and then what chance it had immediately after that action. Using his model, he can quantify how much each player affected his team’s chance of winning during the game.

https://www.afr.com/companies/sport/liverpool-show-moneyball-works-in-soccer-too-20190523-p51qlc ( sorry it's paywalled but very interesting if you can read it).

Seems like a similar approach would be possible for basketball as well. Liverpool use the data in their recruitment strategy , combining it with other information and the financial cost of acquiring the player to try and get the best bang for their buck.

[lessthanjake]

My answer to the question of the thread is basically no, because of a combination of inherently limited sample sizes and the fact that players change in quality relatively rapidly over the years.

Overall, we want to get at impact, since there’s lots of little things on the court that matter but aren’t really specifically quantifiable. IMO, the best we can do basically is to take RAPM and layer on other data as a prior. That data can be box data that is weighted in a way that correlates with RAPM. Now that we have tracking data, it’d be even better to also include tracking data there as well. So basically, you take RAPM with a prior that uses box and tracking data that correlates well with RAPM. That will be a lot less noisy than raw RAPM (which is itself less noisy and more accurate than raw on-off). But even that sort of data is inherently inaccurate. Over relatively small samples (say, for instance, a single season), you’re still going to get a lot of noise because, even with a good prior, the RAPM itself is still super noisy in that sample. Using a prior reduces the noise, but you definitely don’t at all eliminate it. Theoretically, one could deal with that by doing that sort of analysis over a larger time horizon. So, for instance, what if we used RAPM with the same kind of prior, but did it over a 5-year time period, instead of a 1-year time period? That’d be less noisy, because 5-year RAPM is a lot less noisy than 1-year RAPM. But then you run into the problem that players are not the same players over those long periods of time—and that’s true both for the people whose data output you’re looking at but also the teammates and opponents that the RAPM model is aiming to correct for. So you lower the statistical noise but in exchange you get a bunch of substantive imprecision. And that’s not even mentioning the obvious fact that that’s just not an option at all if what you’re specifically wanting to do is compare individual seasons, nor is it mentioning specific methodological flaws or blind spots in a model.

Basically, the problem is that the data we are interested in and the rate at which players change over the years means that we are left with noisy samples and/or have to handwave serious changes in player quality over a larger sample. Both lead to inaccuracy.

That is then exacerbated massively by the playoffs. Playoffs matter more than the regular season, but the samples are even smaller. There’s essentially no way to get a genuinely adequate playoff sample for virtually any player, and there’s certainly no way to do it without pulling from so many years that you’re *definitely* handwaving away player changes over the years. So it’s virtually impossible to get genuinely accurate playoff impact data. And yet as fans we have a general sense that playoffs are what shows what a player is really made of, and that some players are genuinely better or worse in the playoffs than in the regular season. We basically can’t accurately measure any of that though.

So yeah, my answer is generally no. I think the best middle ground on all this is probably to take something like 3-year RAPM with a good box/tracking-data prior that correlates well with large-sample RAPM, and include both regular-season and playoff data but give playoff data some additional weight. I’m not aware of anything that does exactly that, but it’s theoretically possible and would probably be something I’d like. It would still have the types of flaws I identified above, though, and inevitably also some other methodological flaws or blind spots (for instance, no prior is perfect, so it’ll always unduly favor or penalize certain players).

Yea unfortunately Cheema's RAPM data only goes till 2021. So I still haven't seen any complete data-set of RAPM from 1996 to 2024 unfortunately, I'm not complaining btw just trying to understand what is and what isn't available.

I started compiling some players single season peak On-Off and career On-Off from BBR and it's interesting that Draymond's from 2016 is higher than every single other 2000s and onward player in the top 50 or 60 all-time. I understand sample size and taking one data-set with a grain of salt but that is still really interesting.

We do actually have RAPM data that goes through 2024:

https://docs.google.com/spreadsheets/d/1bg8KxzagN7D0O16EmUO9_kCyXwthEUjKywlrWPQUQt8/edit#gid=0

[lessthanjake]

For me, the gold standard would be a stat that is based on feature-extracting skills based on player tracking data. I feel like it's only a matter of time before NBA teams are doing this for particular skills (some will be easier than others), if they're not doing it already, but I'm not what along these lines will be available to the public.

I wonder if anyone is modelling basketball like Liverpool model football.

Instead, he spent months building a model that calculates the chance each team had of scoring a goal before any given action – a pass, a missed shot, a slide tackle – and then what chance it had immediately after that action. Using his model, he can quantify how much each player affected his team’s chance of winning during the game.

https://www.afr.com/companies/sport/liverpool-show-moneyball-works-in-soccer-too-20190523-p51qlc ( sorry it's paywalled but very interesting if you can read it).

Seems like a similar approach would be possible for basketball as well. Liverpool use the data in their recruitment strategy , combining it with other information and the financial cost of acquiring the player to try and get the best bang for their buck.

It’s interesting to think about that sort of approach in basketball. Like, anytime a player gets the ball, you’d measure the expected points scored in the possession (based on player positions, shot clock situation, etc.) and then you subtract that from the expected (or actual) points after they score, pass, or turn the ball over. Similar to how this sort of analysis works in soccer, that’d give an expected value of a player’s on-ball actions.

I think there’s a couple issues with that though:

1. It doesn’t measure off-ball action. That’s a flaw in the soccer version of these sorts of stats as well.

2. It’s not really a type of model that could meaningfully measure most any individual defense. Again, that’s a flaw in the soccer version too, but is perhaps less important there because player roles are more siloed and so you have lots of attacking players in soccer for whom their defensive actions are much less important than any basketball player’s defense. In other words, the thing this sort of model misses doesn’t always matter all that much in soccer, but it would always matter a good bit in basketball.

In general, I actually think impact data in basketball is actually better than this. This sort of analysis is a bottom-up approach that tries to take discrete player actions and model out their value. But it still inherently can’t get at everything a player does. The beauty of impact data is that it is a top-down approach that is inherently looking to measure everything a player does. This is because it is not limited to valuing discrete player actions, but rather is asking what happens when they’re playing compared to when they aren’t, and what happens when they’re playing is affected by *everything* they do rather than just specific discrete actions that a model might look at. The reason you can’t really do this analysis in soccer is basically because players aren’t subbed out very much, so there’s just not much of an “off” sample for most players, which means that you can’t meaningfully do impact data. Without impact data, you’re kind of just stuck doing a bottom-up approach and trying to be as rigorous as possible about how you are valuing player actions. It’s probably the best that can be done in soccer, but I think we have basketball approaches that are better.

[Special_Puppy]

I think we are basically at that point now. You have to know which ones to use and how to weigh them, but you do get most of the picture with just advanced all in one stats