Lacrosse Analytics

D1 Mens Lacrosse
Post Reply
laxreference
Posts: 1099
Joined: Wed Aug 29, 2018 3:54 pm
Contact:

Lacrosse Analytics

Post by laxreference »

With the 2019 trophy safely secured in a case in Charlottesville, I see that the 2020 threads have been spun up. Thought it was about time I did the same.

My first post on the new thread is a look back at how well the Lax-ELO predictions did during 2019 .

There are 541 games in the 2019 database, and the Lax-ELO ratings for each team heading into each contest can be converted into an estimate that each team will win.

The main (representative) question in calibrating the model is something like: for all the teams that were given a 60% chance to win, did they win about 60% of the time. If you extrapolate that out to all the various win probabilities, you can get a sense for whether the model is well calibrated.

Fortunately, there was nothing glaring when I looked at the rates within each bucket.
2019Calibration.png
2019Calibration.png (15.22 KiB) Viewed 10209 times
Data Engineer/Lacrosse Fan --- Twitter: @laxreference --- Informed fans get Expected Goals, the new daily newsletter from LacrosseReference
laxreference
Posts: 1099
Joined: Wed Aug 29, 2018 3:54 pm
Contact:

Re: Lacrosse Analytics

Post by laxreference »

Last year, I built a simulation model that would run through the remaining games, see who won, and then project each team's NCAA hopes. It worked out pretty well, so I decided to try and see what it had to say about the 2020 season. Across thousands of sims, it gives you a sense for how often each team makes the tournament as well as the breakdown of those bids between auto-bids and at-large selections.

For example, last year's champ, UVA, has an 84% chance to make the field. They won the ACC title in 30% of the simulations and were selected as an at-large in an additional 54%. In addition to the raw probabilities, the simulation also projects where each team will fall in terms of RPI, SOS, and Strength-of-Record. The model thinks that UVA will have the 10th most impressive resume, but their SOS (and opponents' SOS) bumps up their average RPI to 8. That is right on the cusp of a seed in the tournament and a first round home game.

Same stats are available for every team.

I also dive into how the RPI vs SOR distinction can play a big role for lower-tier and mid-tier teams.

And lastly, a discussion of how, if RPI is used too heavily by the committee, mid-tier teams are at a huge disadvantage. Richmond is the best example of this.

Full post is here.
Data Engineer/Lacrosse Fan --- Twitter: @laxreference --- Informed fans get Expected Goals, the new daily newsletter from LacrosseReference
User avatar
Matnum PI
Posts: 11101
Joined: Mon Jan 22, 2018 3:03 pm

Re: Lacrosse Analytics

Post by Matnum PI »

Read your post after you posted a link on Twitter. Good stuff. Two questions:

1- Your 2020 predictions are based on the RPI, SOR, etc. from 2019, yes? I guess this is unavoidable for 2020 predictive models?
2- Why would the NCAA need you, or innumerable others who feel the same way, tell them that RPI is a, to be kind, imperfect metric?
Caddy Day
Caddies Welcome 1-1:15
laxreference
Posts: 1099
Joined: Wed Aug 29, 2018 3:54 pm
Contact:

Re: Lacrosse Analytics

Post by laxreference »

One clarification: the only thing we take from 2019 is the ELO ratings that we used to project each game. And even then, these are regressed back toward the average to account for the fact that 2020 is a new season. From there, we run through each game, using the Lax-ELO ratings to project the winner of each game. That gives us a "simulated" season; from there, we calculate RPI and all the other stuff as if it were selection sunday 2020.

The main distinction is that this allows the RPI/SOR/SOS calculations to account for the 2020 schedules. Since schedules are such a big part of the RPI calculation, simply using final RPI from last year wouldn't make sense.

As for your second question, I think it's less about saying RPI is not great. The whole point of that discussion was to point out the ramifications of hewing to RPI. The Richmond example really stood out to me. If you were to use something like SOR, that didn't weight opp's opponent's schedules to the same degree, you wouldn't dock Richmond quite as much for the fact that their conference slate is going to be weak.

And who knows, maybe the committee this year won't stick to RPI so closely? The criticism of it as a metric is irrelevant if it isn't the overriding metric used for tournament selection. Nothing wrong with calculating RPI or having it as part of a basket of metrics they use to select and seed teams.
Data Engineer/Lacrosse Fan --- Twitter: @laxreference --- Informed fans get Expected Goals, the new daily newsletter from LacrosseReference
User avatar
Matnum PI
Posts: 11101
Joined: Mon Jan 22, 2018 3:03 pm

Re: Lacrosse Analytics

Post by Matnum PI »

Or use numerous metrics and be transparent about your rationale for selecting Team A over Team B. And RPI can be part of that rationale. For me, the lack of transparency is the most frustrating part of this process. Unless they came straight out and said, We use RPI. Period. Then I might have a new frustration. :)
Caddy Day
Caddies Welcome 1-1:15
NElaxtalent
Posts: 237
Joined: Mon Aug 06, 2018 10:23 am

Re: Lacrosse Analytics

Post by NElaxtalent »

Excellent insight.

RPI (& esp conf tourneys) stack the deck against non ACC/B1G teams.

SOR > RPI
wgdsr
Posts: 9471
Joined: Thu Aug 30, 2018 7:00 pm

Re: Lacrosse Analytics

Post by wgdsr »

Matnum PI wrote: Thu Jan 23, 2020 2:37 pm Or use numerous metrics and be transparent about your rationale for selecting Team A over Team B. And RPI can be part of that rationale. For me, the lack of transparency is the most frustrating part of this process. Unless they came straight out and said, We use RPI. Period. Then I might have a new frustration. :)
they did say that. last year.
of course, to your point 2 of the problems are ---
1) you don't in fact know what you're going to get every year.
2) the nc$$ selection rules list the primary criteria to be used, and rpi isn't one of them.
wgdsr
Posts: 9471
Joined: Thu Aug 30, 2018 7:00 pm

Re: Lacrosse Analytics

Post by wgdsr »

laxreference wrote: Thu Jan 23, 2020 2:01 pm Last year, I built a simulation model that would run through the remaining games, see who won, and then project each team's NCAA hopes. It worked out pretty well, so I decided to try and see what it had to say about the 2020 season. Across thousands of sims, it gives you a sense for how often each team makes the tournament as well as the breakdown of those bids between auto-bids and at-large selections.

For example, last year's champ, UVA, has an 84% chance to make the field. They won the ACC title in 30% of the simulations and were selected as an at-large in an additional 54%. In addition to the raw probabilities, the simulation also projects where each team will fall in terms of RPI, SOS, and Strength-of-Record.
alas, you may need to recalculate.
there is no auto-bid for the acc title. more importantly, that pulls an at large away, potentially.
User avatar
HopFan16
Posts: 5577
Joined: Tue Jul 31, 2018 1:22 pm

Re: Lacrosse Analytics

Post by HopFan16 »

NElaxtalent wrote: Thu Jan 23, 2020 3:05 pm RPI (& esp conf tourneys) stack the deck against non ACC/B1G teams.
Not really. If you schedule tough opponents, it will show up in your RPI, even if you're not in one of those conferences. Penn routinely has a top 10 RPI (#5 last year) because they have a hard schedule. Loyola and Towson were both top 10 RPI last season. Cornell, Denver, and Army weren't far off. All of those teams were ahead of UNC and Rutgers, for instance. It wasn't too long ago when Albany had back to back years being in the top 5 of the RPI. Pretty sure even Marquette was in the top 10 a couple years back. Playing in the B1G/ACC is not a guarantee of a top RPI. Just like playing outside of them does not doom a team to RPI jail. Play better teams. Not the fault of the B1G or ACC that both their conference and non-conference games are difficult.

High Point figured it out. They were #20 in RPI in 2019 which will never be enough for an at-large bid no matter how many people complain. So this year they went and added Maryland and Cornell to the schedule (and removed St. John's). Maybe that gives them a fair shake this year. At least they're trying.
laxreference
Posts: 1099
Joined: Wed Aug 29, 2018 3:54 pm
Contact:

Re: Lacrosse Analytics

Post by laxreference »

HopFan16 wrote: Thu Jan 23, 2020 3:20 pm

High Point figured it out. They were #20 in RPI in 2019 which will never be enough for an at-large bid no matter how many people complain. So this year they went and added Maryland and Cornell to the schedule (and removed St. John's). Maybe that gives them a fair shake this year. At least they're trying.
High Point is a great example. They did vastly improve their schedule this year. The problem for them is that with all those enhancements, their schedule still projects as the 33rd toughest in D1. Playing a full SoCon slate guarantees that your schedule can be no better than average. Richmond is in the same boat.

I think the key point here is that ACC/Big Ten/Ivy teams can either a) fill their non-conf slate with tough opponents and have a top 10 SOS or b) play cupcakes and have a middling SOS. A team like High Point can schedule a really tough non-conf slate, and that only gets them to the 33rd toughest schedule. For those teams, the margin of error is basically zero.
Data Engineer/Lacrosse Fan --- Twitter: @laxreference --- Informed fans get Expected Goals, the new daily newsletter from LacrosseReference
NElaxtalent
Posts: 237
Joined: Mon Aug 06, 2018 10:23 am

Re: Lacrosse Analytics

Post by NElaxtalent »

HopFan16 wrote: Thu Jan 23, 2020 3:20 pm
NElaxtalent wrote: Thu Jan 23, 2020 3:05 pm RPI (& esp conf tourneys) stack the deck against non ACC/B1G teams.
Not really. If you schedule tough opponents, it will show up in your RPI, even if you're not in one of those conferences. Penn routinely has a top 10 RPI (#5 last year) because they have a hard schedule. Loyola and Towson were both top 10 RPI last season. Cornell, Denver, and Army weren't far off. All of those teams were ahead of UNC and Rutgers, for instance. It wasn't too long ago when Albany had back to back years being in the top 5 of the RPI. Pretty sure even Marquette was in the top 10 a couple years back. Playing in the B1G/ACC is not a guarantee of a top RPI. Just like playing outside of them does not doom a team to RPI jail. Play better teams. Not the fault of the B1G or ACC that both their conference and non-conference games are difficult.

High Point figured it out. They were #20 in RPI in 2019 which will never be enough for an at-large bid no matter how many people complain. So this year they went and added Maryland and Cornell to the schedule (and removed St. John's). Maybe that gives them a fair shake this year. At least they're trying.
Actually it won't. Read the SoCon/Richmond section. There is no way to remove your weak conference games. Richmond's OOC SOS is incredible but the 4 games vs the lower part of the SoCon offset that OOC schedule.
wgdsr
Posts: 9471
Joined: Thu Aug 30, 2018 7:00 pm

Re: Lacrosse Analytics

Post by wgdsr »

NElaxtalent wrote: Thu Jan 23, 2020 3:46 pm
HopFan16 wrote: Thu Jan 23, 2020 3:20 pm
NElaxtalent wrote: Thu Jan 23, 2020 3:05 pm RPI (& esp conf tourneys) stack the deck against non ACC/B1G teams.
Not really. If you schedule tough opponents, it will show up in your RPI, even if you're not in one of those conferences. Penn routinely has a top 10 RPI (#5 last year) because they have a hard schedule. Loyola and Towson were both top 10 RPI last season. Cornell, Denver, and Army weren't far off. All of those teams were ahead of UNC and Rutgers, for instance. It wasn't too long ago when Albany had back to back years being in the top 5 of the RPI. Pretty sure even Marquette was in the top 10 a couple years back. Playing in the B1G/ACC is not a guarantee of a top RPI. Just like playing outside of them does not doom a team to RPI jail. Play better teams. Not the fault of the B1G or ACC that both their conference and non-conference games are difficult.

High Point figured it out. They were #20 in RPI in 2019 which will never be enough for an at-large bid no matter how many people complain. So this year they went and added Maryland and Cornell to the schedule (and removed St. John's). Maybe that gives them a fair shake this year. At least they're trying.
Actually it won't. Read the SoCon/Richmond section. There is no way to remove your weak conference games. Richmond's OOC SOS is incredible but the 4 games vs the lower part of the SoCon offset that OOC schedule.
actually, it probably should. duke plays 5 socon teams. uva plays 4 plus utah. unc plays 3 of the lower part of the socon schedule as well as lafayette and dartmouth.
no one plays 15 tough games. you have to play enough of them. and if you have 3-4 that are "too weak", your overall record has to reflect that. there's plenty of crossover with the acc.
then you get the advantage of having a better shot of winning your auto bid as well. high point and richmond are in a legitimate spot if they can win some games. high point probably has to lose st bonnies at some point soon if they're serious about it, but by contract may have to wait until next year.
NElaxtalent
Posts: 237
Joined: Mon Aug 06, 2018 10:23 am

Re: Lacrosse Analytics

Post by NElaxtalent »

The kicker though is when those ACC/B1G teams "get" to face 2 extra top-tier RPI teams in the conf tourney. Those tourney games turbo-charge their RPI.

That said, the ACC/B1G/Ivy conferences clearly deserve multiple bids each year. But as the OP cited, the mid-tier conferences have to virtually thread-the-needle for an AL bid.
wgdsr
Posts: 9471
Joined: Thu Aug 30, 2018 7:00 pm

Re: Lacrosse Analytics

Post by wgdsr »

NElaxtalent wrote: Thu Jan 23, 2020 4:46 pm The kicker though is when those ACC/B1G teams "get" to face 2 extra top-tier RPI teams in the conf tourney. Those tourney games turbo-charge their RPI.

That said, the ACC/B1G/Ivy conferences clearly deserve multiple bids each year. But as the OP cited, the mid-tier conferences have to virtually thread-the-needle for an AL bid.
not every advantage is supposed to be for the sheep.
and all but one of the teams in the conference tourney takes a loss, half of them in their only game. the ones that take a loss only will get nicked, not helped, with the new rpi-or-bust rules. sos used to be bolstered.
and if you're in a conference that doesn't have at least one other tough team to play (say... in the finals), then you should win the conf tournament no probs, right?
simply put -- if you have easier games in conf, it's balanced by a better record (and better auto bid chances), if in fact you couple it with a challenging ooc schedule.

loyola has more conference games than any socon team. they likely would've qualified at large in each of the last 4 years. towson is similar but in a smaller conference.
User avatar
Matnum PI
Posts: 11101
Joined: Mon Jan 22, 2018 3:03 pm

Re: Lacrosse Analytics

Post by Matnum PI »

Caddy Day
Caddies Welcome 1-1:15
User avatar
Matnum PI
Posts: 11101
Joined: Mon Jan 22, 2018 3:03 pm

Re: Lacrosse Analytics

Post by Matnum PI »

Matnum PI wrote: Thu Jan 23, 2020 5:51 pm laxreference, how do you tabulate your SOS?

Image
ELO. Ignore me...
Caddy Day
Caddies Welcome 1-1:15
spartanslynx
Posts: 189
Joined: Mon Oct 07, 2019 5:31 pm

Re: Lacrosse Analytics

Post by spartanslynx »

Nice read! I agree about the selection problematic. Looking forward to follow the ELO and game Winning precentages.

In the formula about individual performence I feel like FOGOs gets too high scores. Counting both FO wins and GBs feels like potato on potato. And no negative to FO losses. Are you satisfied with the strength of FOGOs in that list?
laxreference
Posts: 1099
Joined: Wed Aug 29, 2018 3:54 pm
Contact:

Re: Lacrosse Analytics

Post by laxreference »

spartanslynx wrote: Thu Jan 23, 2020 7:14 pm In the formula about individual performence I feel like FOGOs gets too high scores. Counting both FO wins and GBs feels like potato on potato. And no negative to FO losses. Are you satisfied with the strength of FOGOs in that list?
Thanks for the kind words; it's exciting to be back at it for another season.

As for the FOGO EGA domination, I'm happy with it. I mean, FOs are incredibly valuable. The rate at which they lead to goals for the team that picks them up is very high; much higher than a normal GB. Remember, the EGA calcs are all based off of the average "value" of each type of play. If FOs didn't lead to goals as often, FOGOs wouldn't have as much value and they wouldn't show up in the list as often. I can understand if your argument is: "I wish that face-offs weren't as important as they are." That is up for debate.

The one thing that would be nice is to be able to distinguish between FOGOs who are good at winning at x and generating a fast break vs those that tend to win scrums that don't lead to the same offensive advantage, but like so many other things that would be nice, it's impossible to glean from the pxp data logs.

And to your point about double-counting, I believe I give the FOGO half-credit if someone else picks up the GB.
Data Engineer/Lacrosse Fan --- Twitter: @laxreference --- Informed fans get Expected Goals, the new daily newsletter from LacrosseReference
laxreference
Posts: 1099
Joined: Wed Aug 29, 2018 3:54 pm
Contact:

Re: Lacrosse Analytics

Post by laxreference »

In my post last week, I presented the current projections for each team, with a deeper-dive into just the Big East, ACC, and SoCon. So here are the rest of the conferences.

There are definitely some interesting things to think about in this analysis, namely, how teams are trying to balance the dual concerns of record and schedule strength. It almost seems like some of the teams that have recently been bubble teams over-learned the lessons of not having enough schedule strength. Where there are some stronger teams that seem to have gone the opposite direction.

Given that schedules aren't made out as far in advance as a sport like football, it does make sense that we might see schedules that are more responsive to the final results of the prior year. Interesting thing to keep an eye on.
Data Engineer/Lacrosse Fan --- Twitter: @laxreference --- Informed fans get Expected Goals, the new daily newsletter from LacrosseReference
spartanslynx
Posts: 189
Joined: Mon Oct 07, 2019 5:31 pm

Re: Lacrosse Analytics

Post by spartanslynx »

Nice read!

How Penn State and Maryland play in the conference can decide a lot. Look at last year with Hopkins.
Post Reply

Return to “D1 MENS LACROSSE”