Print

Add to Reputation:  

What do you think of this post?

I agree I disagree  
Comment:

[VC]

Points-based ranking systems suck.  I know you clan kids think your arbitrary shit that gives +10 for Most Professional and a bonus point for a headshot are all happy with your website stats shit, but you know what?  I think it's a lot of crap.

First of all, a direct points system doesn't account for more playing = more points.  A great player is ranked poorly for not hanging around in the server all day.

Second, while there is some relationship observed between the scores of the fragging and fragged players that limits high-ranked players to something like +2 for a kill while noobs and walking failures can steal 10 or 25 for getting lucky with a Klobb, it's still pretty arbitrary.  How do you know that 10-point kill was five-times tougher than a 2-point kill?  You don't.


Since my ambitions include making my own games, I realized that would be a good opportunity to add gamer skill rankings that aren't stupid.


In the recent "omg headshots are too good" thread, I mentioned that I copied off my console log after an evening of playing.  I did so to test my ranking system.  It seems to work pretty well:



The coolest part is that the computation of rank change when a player beats another can be implemented with only a few lines of code.  Plus, its responsiveness is adaptable.  Here it's on 40:1.  At 10:1, it really just shows who's on a hot streak and who is not, and at 100:1 it becomes much more smooth, but would take far more than a hundred frags to become representative of a player's true skill.  Thus, I figure I can adapt this to be useful in games for per-round, per-session, and lifetime stats.


At some point I should stop dicking around with colorful graphs and get back to work on making classic maps for this "Golding Eye" thing I've heard so much about recently.


So, anyone with a server have a really big ass server log that I could process to further my testing?

[major]

Do you need the logs of servers with stats? Or just straight kill logs?


If kill logs, here you go: http://wcservers.net/Goldeneye/14DevServerLogs.zip

   ^245 long/medium server logs(since launch on DEV server)

[kraid]

Sure, the current stats systems are not really messuring the skill, if you can find a better way of doing this, we all would be pleased.

[VC]

Do you need the logs of servers with stats? Or just straight kill logs?

What I did was all pre-processed by Linux classics, cat, grep, cut, paste, sed.  Brotha's gonna work it out.

[Mangley]

Like VC said HLstatsx is just an arbitrary points system. It's bad enough in standardized form in games it's suited to like CS and TF2 where it almost works.

Adapted to GE:S, as seen throughout Beta 3 and 4, HLstatsx simply leads to people statwhoring, spawn killing and only playing certain game modes where they're guaranteed to excell in points.

[Entropy-Soldier]

GE:S can't really have an accurate ranking system because of the availability of armor and frequency of spawn killing.

[VC]

Let's see, 2414 players, 53454 kills.

With responsiveness at 100:1, which seems about right, skill ratings for Major's logs come out thus.

#Rank, Skill factor: Name
#1 - 81: WNxNalle
#2 - 66: a_puma01
#3 - 64: Drumman1000
#4 - 62: pengfx
#5 - 62: RC | Viper
#6 - 60: [TW]DrGuppy
#7 - 59: ToxicS |LAB|
#8 - 58: Black Dynamite
#9 - 57: Blindside
#10 - 56: parian
#11 - 56: CnB| Yiggs
#12 - 55: Darius I
#13 - 55: MaJoR
#14 - 54: raisin. Maybe. I don't know.
#15 - 54: ☃ ʷigglєs ☃
...and the worst player to have a registered kill...
#2174 - 0.85: Cthuloo

Major, having actually played on the server, does this seem like an accurate evaluation?

GE:S can't really have an accurate ranking system because of the availability of armor and frequency of spawn killing.

I don't see how armor and spawn-kills matter to a ranking system since it affects everyone equally and thus is zero-sum unless someone takes advantage and thus (hold on to your seat) performs better.

[major]

Only people I really remember was yiggs and Darius. We all were battling pretty good, so our score being similar would make sense. Nalle is pretty damn good, so that also fits the bill.

Logs from E-S or Wnx would be best as regular players play in there so we could judge better.

[Entropy-Soldier]

Well, by saying that you claim to have no issue with someone picking up armor several times in the middle of a fight just so they won't lose a skill point.  GE:S is set up in such a way that if someone knows how to camp the armor and only plays when the conditions favor that approach, then they can not possibly ever die, with the exception of a few scarce circumstances such as everyone in the server getting fed up with their antics and rushing them at the same time.

That being said, some people can camp the armor better than others.  However, the fact that stats encourage armor camping and spawn killing should immediately tell you that it's a bad idea.  You don't want every single player in the match trying to camp the armor because then it becomes a matter of who got in the game first and who can aim reasonably well.  Everyone who was unlucky enough not to get an armor spot gets to enjoy spawning and dying the whole match.  Stats don't always favor the better player, more often than not it's simply who's willing to do more for extra points.

[VC]

Well, by saying that, you are exposing a perspective that YOU feel that the CORRECT way to play Goldeneye is to IGNORE DEATHS by aggressively attacking at all times.

Player skill, as I see it, should be representative of a player's expectation to be the winner in a random matchup.  Indeed, that's what chess/ELO scoring is supposed to be about.

If a player "whores" the stats by only playing Facility and garding the armor, then he will have a high rank.  And guess what, while you won't see him often, when you do run into him, you can probably expect that he will kill you before you kill him.  Hence, the high ranking is representative of the actual situation: prepare to die, because he's got a bigger gun and thicker skin than you do.

But still this does not invalidate the stats as I calculate them.  Okay, A.C. Ten is in his corner counting seconds until the armor respawns.  Meanwhile, throughout the rest of the map, everyone else is dueling with mid-tier weapons and little/no armor, and their stats are being calculated the same way: he who wins most of his  duels gets a powerful rank.

At this point you could say that AC-Ten and Rambo have about the same rank, but that AC needs to whore the armor while Rambo can do it without pickups, so Rambo should rank higher.  Well, maybe, but if he's truly a better player, he should be able to compete for the armor too.  Until he proves it by kicking AC's ass, he hasn't earned that additional rank.  Otherwise, both are performing equally well in combat under parallel conditions.

A guy who just spawned in can Klobb me after I take 180 damage from an RC-P90 guy and get the same point that the RC-P90 guy almost earned; should we pro-rate frags, or base our results on RESULTS?


Besides, Dr. No Amor, Dr. No Problem.

[Entropy-Soldier]

But the stats should be telling me what would happen in an equal match up, where both players have access to the same weapons and are at the same health.  If Rambo and Mr "I camp the armor and only play with new players" competed 1v1 on complex, the result would obviously contradict the ranking system that claims the two players are equal.

I do capitalize on the armors benefits myself, but there's a difference between picking it up frequently and sitting so close to it that anyone that attacks me needs to go through two blue gauges just to touch my life bar.  When you throw in the fact that your proposed ranking system doesn't even note the people that one is playing against, it's easy to only join the server when completely new players are on.  The stat whore would get ahead quickly, while the legitimately good player who favors competition would suffer due to competent opponents and the high possibility of spawning next to someone with a good weapon and good aim.

[VC]

"But the stats should be telling me what would happen in an equal match up, where both players have access to the same weapons and are at the same health."
The rank is no longer valid because it does not reflect Goldeneye Source As Played On This Server, but rather some fantasy land where everyone stands back-to-back before walking ten paces.  You're focusing on one element of gameplay (the all-things-even duel situation) at the cost of ignoring the whole of the game.

"it's easy to only join the server when completely new players are on."
My formula does compare the players' current stats.  A high-ranked player defeating a low-ranked player is under-weighted.  That's part of getting away from the HLstatsX points system; if your rank is high and you beat a first timer, you don't get much/anything from it because that player has little/no record to prove that you actually overcame a challenge.

Let's look at the very first graph again, but at 10:1 responsiveness, which really shows real-time hot/cold-streaks instead of long-term performance.



My line (purple) quickly approaches Perfect at first, because it was basement and I was simply not dying against newbie players.  (I got my no-deaths acheivement and two 6-Awards in this session.)

Someone (red) showed up who knew how to fight, and my inflated -by-newbies rank is quickly normalized until I adapt my strategy and start winning my fights again.  Then a third player (olive) entered and started losing against me but winning against Red until the end when he adapted to Red.


Beating up newbies to inflate rank is impossible to avoid since, if the population of gameplay events is one guy beating up newbies, then he's the best player in the pool by a wide margin.  You seem to want to force all players to compete against a hypothetical metric, a perfectly matched opponent HP/AP/Arsenel-wise but with flawless gameplay as the standard.  But that both does not exist and is not seen in the actual game.

Anyway, I'm tired of arguing this.  Your vision of how players should be ranked and my vision of a way that they can be ranked practically and usefully differ.

If you wish to participate further, you could throw me some server logs.  If you have a large number of regulars with whom you are well-familiar, you could look over my results and provide feedback to tell if my evaluation agrees with how you would sort them, despite not taking AC-10 and spawnkilling into account.

[Entropy-Soldier]

This was never about the best ranking system, it's simply about the fact that ranking systems don't work in regards to Goldeneye: source.  I made one quip about yours in particular because it provided contrast to HLstats, as it forgoes the point system and simply using the base data to calculate ranking.


Stats systems are inaccurate, encourage unfair play, and just motivate everyone to view their point total/skill index as proof of their ability rather than evidence of their stat whoring.  No matter how you set it up, stats just encourage people to exploit game mechanics in their favor rather than challenge themselves and enjoy the experience.


Regardless, i wasn't aware this was an argument.  I was simply giving my input that was gained over a collected 1000 total hours playing beta 4 in different servers with and without stats.

[VC]

I was arguing that such stats can be a valid metric against your assertion that games that allow players to have varied amounts of health and do not have a 3, 2, 1, FIGHT countdown make objective player evaluation impossible.

[V!NCENT]

VC, people claim that you are pro at GES, right? Have you tried going to ges-stats.com and sorting on skill? You're up high

But I like your project a lot! Example; yesterday I was pwning the shit out of everyone at wnx uk. I scored a lot of headshots and won the rounds easily, untill the wnx crew came over and I no longer ruled. I could disconnect and wait for easyer battles and not lose headshot counts and points. There is the obvious flaw with the current system.

[VC]

There's really nothing one can do about that using a real-time system that only keeps track of one data point (current rating) per player.  Your comment does make me wonder if I could make a variant that can adjust player weightings when complete historical stats are available... that might make E-S happy, but I won't get my own hopes up.

[Entropy-Soldier]

Regardless of what i think of that, I'm sure we can both see this is going nowhere because, as you said, we have conflicting ideas of what a stat system should measure.  Stats do convey accurate data in regards to what normally happens on the server, so if someone has a high ranking they would have at least had to play well.  That's not to say it's an accurate calculation of player skill as i see it, but it does tell you who normally has the better performance.

Of course, it may be worth keeping in mind that HLstatsX isn't designed to be accurate, its purpose is to keep people coming back to the server.

[jjmusicnotes]

Perhaps VC could implement a weighted ranking system that sorts players based on who fights dirty and who fights honorably.  Of course, KM would have to generate some sort of code to determine inherent moral parameters of players.

Might I suggested an "alignment" bar circa Black and White?

[VC]

HLstatsX isn't designed to be accurate, its purpose is to keep people coming back to the server.

jj, don't get too excited.  I designed this as a generic ranking system for my own projects.  That doesn't mean it couldn't turn into a GES ranking server mod that looks at details like HP/AP and Honorability someday, but this is a personal weblog entry, not a GES-development entry.

I used GES combat data because it is accessable: play games, copy console before closing, paste into notepad, throw at bash script, run program, open results with LibreOffice, make pretty graph, have a Kit Kat bar.

[jjmusicnotes]

Haha, I was mostly goofin' on the throwback between you and ES by hyperbolically suggesting solutions.

Moreover, in a more honest sense, I do like the scheme of your ranking system.

[Kiihhu]

I know this is an old topic, but I just read it and noticed a lot of discussion about armor whoring. Couldn't we just implement it like it was in the original game, so that the armor wouldn't spawn if there's someone standing too close to it? That would at least lessen the armor whoring and perhaps prevent grabbing the armor in the middle of a fight.

[VC]

Old topic, old argument.  Adding a radius just means campers will game the radius.  They can stand near it to hold it until it is safe, then step away to spawn it and take it.  With timed respawn, they can be pushed off of it by an aggressor and lose control.

Thanks for reminding me about my stats project though; it got pushed aside and half-forgotten.  Appends to TODO list.