Ayrton Senna, McLaren, Adelaide, 1989

Brawn and Smedley explain how they calculated ‘F1’s fastest driver’

F1 history

Posted on

| Written by and

Naming Ayrton Senna as the fastest driver ever to have raced in Formula 1 is not exactly a controversial opinion.

The three-times champion was a renowned master of the flying lap. He took his first pole position in 1985, and just four years later equalled Jim Clark’s then-record of 33 poles.

By the time Senna met his own untimely end in 1994, he had almost doubled that tally to 65 pole positions from 162 appearances. That record eventually fell, to Michael Schumacher, but it took the German ace until his 233rd appearance to equal Senna.

Their names appeared first and second in a list of the fastest Formula 1 drivers ever as determined by official research produced by the sport in conjunction with sponsor Amazon Web Services. Lewis Hamilton – who claimed the all-time pole positions record in 2017 – makes a similarly uncontroversial appearance behind them in third place.

But the ranking of several other drivers on the list produced debate and even derision. Heikki Kovalainen ahead of Sebastian Vettel? Jarno Trulli quicker than Alain Prost?

Few would dispute Senna’s position at number one…
“I don’t think they’re laughing at it, I think it’s caused some friendly debate,” said Ross Brawn, Formula 1’s managing director of motorsport, in a video conference interview with media including RaceFans yesterday.

“I think when you understand the methodology then I think people will start to understand.” However he admits the results which have been produced are “controversial”.

Brawn stresses the list is intended to reflect only the ultimate one-lap performance of drivers, and no other aspect of their abilities. It was produced by examining qualifying performances since 1983 between pairs of team mates as they, theoretically, have the same equipment at their disposal and therefore form a good basis for comparison.

“We look at two team mates, exactly the same day, same situation, same opportunity,” Brawn explains. “And we set a time increment between those two team mates and you build that up over time to see how that’s averaging.

Advert | Become a RaceFans supporter and go ad-free

…but is Kovalainen really F1’s eighth-fastest driver ever?
“Then you then need one of them to migrate to another team. [They] go to another team. So Driver A’s quicker than Driver B, Driver B goes to another team and he’s quicker than Driver C. So you can say that A is quicker than C because A beat B in his own team and B’s beaten C in another team.

“Then you build that out with enough information and analysis to start to understand who are the quick guys and who consistently out-performed their team mates and by how much.”

Brawn, who guided the likes of Ferrari and Benetton to championship success and laid the groundwork for Mercedes to become the all-conquering machine it is today, says similar research is conducted by teams to inform their decisions on which drivers to sign. Such as at Mercedes in 2012, when Hamilton was hired to replace Schumacher.

“When I was at Mercedes the strategy guys there did a lot of analysis on drivers. We got Lewis after Michael, and that was obviously a very simple decision, but because we didn’t know we could get Lewis we did an awful lot of analysis of other drivers.

“We actually came up with Nico Hulkenberg when we were trying to sign Lewis as our back-up. James Vowles did all the analysis of Nico’s career and what he was like compared to his team mates, his consistency of lap times in the race, even going as far as trying to understand if he was harder on tyres or easier on the equipment.”

Jenson Button, Brawn, Nurburgring, 2009
‘Fuel-corrected’ qualifying times weren’t used
Rob Smedley, who worked alongside Brawn at Ferrari during the peak of the team’s success, and is now F1’s director of data systems, admitted there are some limitations to the model which they had to work around. For example, between 2003 and 2009 drivers carried the fuel loads they would start the race with during qualifying. Therefore their lap times could vary as a result of differences in fuel weight.

This was not taken into account in the model, as Smedley explained. “We took a decision that Formula 1 of that era ‘was what it was’, to put it in layman’s terms.

“So when we’re talking about different fuel loads, if you talk about the period in the early 2000s when we had to qualify with the race fuel, we haven’t normalised for that. And the reason why we haven’t normalised for that is – we looked at this in a reasonable amount of detail – if you look between team mates then the median difference was, I think, 1.5 laps [of fuel] and in 1.5 laps with that era of cars, you’re probably in the region of around about 40 milliseconds difference.

Advert | Become a RaceFans supporter and go ad-free

“Don’t forget, we’re not just taking into account a single race here. The median difference over a season between team mates is zero because sometimes one driver has got more fuel in and sometimes one driver’s got less fuel.”

Other potential influencing factors such as differences between tyre compounds and engine performance settings could also be safely discounted, Smedley says.

“Tyres you don’t really need to consider, which is the beauty of taking the qualifying lap because even in the current format we can always ensure that everybody’s on the softest or the quickest tyre.

“It’s the same with with engine modes which have become a bit of a tool to be used not only in this era but in previous eras as well. If you go back to pre-parc ferme when we had quali engines, there was a difference between the quali and the race engine. There’s a difference between the modes that you would run in qualifying and the race if you look at this area. But again, it all normalises itself.”

Lando Norris, McLaren, Circuit de Catalunya, 2020
Inexperienced Norris ranked highly
One of the more surprising outcomes of the model was the comparatively high ranking of less experienced drivers such Lando Norris, who started his 27th race last weekend. Smedley admits it’s harder to have confidence in the rankings of drivers they have less data on, but says it doesn’t necessarily follow that more experienced and famously rapid drivers such as Juan Pablo Montoya should be ranked higher.

“If you take a driver who’s at the very advent of his career who’s had very few seasons under his belt, like Lando, obviously we would have less confidence in that result than if exactly the same driver was placed there in the year 2030 having had 11 or 12 years in Formula 1. That’s just mathematics and we can’t get around that.

“But because we don’t want to tamper with the result – there are some ways in which we can tune the model to account for the fact that a driver is in his first season, which we do – but you can’t really go any further than that. If the results are that it places Lando in the top 20 and Juan Pablo not, then because this is a very objective exercise in trying to answer a subjective question, it would be wrong for any of us to go and tamper.

“Because if you think that somebody’s personal leaning is that the Lando and Montoya pairing is wrong, somebody else’s personal feeling will be different about another driver pairing. So it’s difficult for us to go in and get this to agree with everybody’s opinions. I don’t think it ever could. All we can do is just present a really objective methodology and then explain the process behind that methodology.”

Go ad-free for just £1 per month

>> Find out more and sign up

At present the study is limited to the last 37 years of a sport which spans seven decades. Smedley says “it’s definitely planned that that we want to go further back” through F1’s history, but the limitations of the available data prevented that for this first iteration of the study.

“If you go back even further into the seventies and the sixties, we’re into essentially curated, handwritten data.

Trulli was “quick… but not on a Sunday”
“That’s an exercise that we want to absolutely move it to, I don’t really want to give you a timeframe on that in terms of when we’ll execute on that, but it’s definitely something we want to do. But that was just going to be time prohibitive to get the first version out.”

As for some of the more controversial rankings in the list, Brawn and Smedley believe they are vindicated by real-world experiences of the drivers.

“I was talking to Steve Nielsen who worked with Trulli at Benetton/Renault,” says Brawn. “He said he was incredibly quick, he was just stunningly quick, he just couldn’t put more than five laps together which is why he never became world champion.

“It’s very important to understand I’m not criticising Jarno, what I’m saying is that this is the fastest driver and the fact that some of these guys were not world champions was because the rest of the skillset wasn’t as strong. It’s the same with Kovalainen if you look at his team mates he had a very strong qualifying record against his team mates.”

Smedely concurs with the verdict on Trulli. “I worked with him at Jordan,” he says. “He was incredibly quick. But not on a Sunday.”

The ranking has prompted a response from drivers already, Smedley adds. “I’m not going to give you any names, but I would say probably 10 of the current grid and probably 10 of the not-current grid.

“I am popular in a few places but, I would say, predominantly unpopular at the minute with most of the people on the grid…”

Formula 1/AWS ranking of the fastest drivers since 1983

RankDriverGap to fastest (seconds)
1Ayrton Senna0
2Michael Schumacher0.114
3Lewis Hamilton0.275
4Max Verstappen0.280
5Fernando Alonso0.309
6Nico Rosberg0.374
7Charles Leclerc0.376
8Heikki Kovalainen0.378
9Jarno Trulli0.409
10Sebastian Vettel0.435
11Rubens Barrichello0.445
12Nico Hulkenberg0.456
13Valtteri Bottas0.457
14Carlos Sainz Jnr0.457
15Lando Norris0.459
16Daniel Ricciardo0.461
17Jenson Button0.462
18Robert Kubica0.463
19Giancarlo Fisichella0.469
20Alain Prost0.514

2020 F1 season

Browse all 2020 F1 season articles

120 comments on “Brawn and Smedley explain how they calculated ‘F1’s fastest driver’”

  1. I was a kid when Fisichella was around. Was Fisichella that quick over one lap? I think this may come down to his stint in Renault and the pole position with Force India.

    1. Yeah, he was certainly amongst those who’d often qualify better than the car was able to go in the races @krichelle.

      I guess I am glad they gave somewhat more of an explanation, but I still think they have to adjust their data and assumptions.

      One huge factor it doesn’t seem to take in account is that drivers develop over the years. So Rosberg more often than not beating Schumacher at Mercedes might have helped him higher up, and in effect then would also help Hamilton higher up, since he often beat Rosberg, who beat Schumacher, who earlier beat Barrichello, Irvine, Piquet etc. But surely Mercedes’s schumacher was not as fast as Ferrari Schumacher. And Benetton Piquet was not the same guy who had won 3 championships in earlier years either (and it doesn’t take in account drivers NOT having the same equiment either).

      I think it skews the model, because otherwise surely we would have more top guys from the 1990s and at least a some from the 1980?

      1. @bascb it would certainly go a long way towards explaining why the data set was so biased in frequently having pairs of drivers appearing if it relied so heavily on relative deltas between drivers.

        There are some assumptions within that data set that are questionable, such as the assumption that, in seasons where fuel load was a factor in qualifying, it would average out over the course of a season and could be ignored.

        There are definitely cases where teams would tend to give one driver a higher fuel load much more frequently than their team mate – indeed, for the 2009 season we should be able to quantify that fuel effect as the weight of the car, with fuel, was published after qualifying.

        The assumptions about equality of cars is problematic when we know there have been teams that have prioritised the lead driver in their team. Similarly, some third party suppliers were accused of aiding certain drivers – for example, during the 1980s, Honda faced complaints from multiple drivers that they were biased (Rosberg, Mansell, Piquet and Prost all made complaints, and I believe that the journalist Nigel Roebuck has said that the head of Honda’s engine division explicitly told him that they were favouring Senna during the Senna-Prost years).

        The assumptions of “if driver A has beaten driver B, and driver B beats C, then A must be faster than B and C” is an assumption that has also broken down – instances of drivers A and C then being partnered together, only for driver C to then beat driver A, are not unknown. Now, some of that may be age corrected and perhaps can be partially corrected for, but it shows that simple comparative models can run into problems.

        Overall, I would have to say that this really looks more like, at best, it might be the starting points for a more exhaustive study – but, in its current form, it looks like it has a number of flaws and is potentially too simplistic in the approach it has adopted.

        1. Age was factored in, according to the official site.
          I can see why they didn’t fuel-correct the lap times between 2003 and 2009, but it may indeed have affected the ranking of some drivers. It would be nice to see if there are drivers that consistently qualified heavier than their teammates. Maybe they are right that it evens out in the long run, but maybe there are some exceptions.
          Driver triangles are not too uncommon. F1 metrics mentions Hill-Villeneuve-Frentzen as an example. This happens because driver performance varies from year to year. The model tries to estimate their average level.

          1. @f1infigures so age is being factored in to at least some extent then, which seems reasonable – but it then raises the question of how exactly they are correcting for age in their assessments.

            As you note, driver triangles are not uncommon, but the question then comes about of how the study untangles that. As others have noted, the question then also comes about on how the algorithm was calibrated against known data and how it then performs when you try to extrapolate outside of those bounds.

            I do also wonder how exactly they manage to account for times when a driver might only have had a small number of team mates, or where they have competed against a team mate with few, if any, references. Toro Rosso is something of an issue in that respect, as there are instances where the only reference that one driver has is to another driver within his same team – which then begs the question of how exactly you compare them.

            I do think that the over-prevalence of paired drivers and the strong bias towards very recent drivers does suggest that there are potential issues with the calibration of the models and some of the assumptions made about the way in which results can be compared, and thus would prefer a clearer explanation of the way in which factors were accounted for (i.e. showing how exactly age was accounted for in their models, for example, rather than just stating it was included).

        2. <>

          What I’ve seen the Honda boss quoted as saying was that Senna instilled a huge loyalty among the Honda engineers since before his time at McLaren, so if you asked who the Honda engineers were rooting for, the answer would likely be Senna. It’s a big difference from favoring Senna with better engines and I believe he fervently denied that there was any material difference in the equipment.

          1. Dan, the exact details of whether or not there were material differences is going to be hard to prove these days, but it was to point out that those allegations were pretty frequently made in that period.

            I believe Ron Dennis has confirmed that Honda, apparently after being lobbied by Senna, tried to have Prost expelled from the team during the 1989 season to give Senna an unopposed run to the title that year, in much the same way that Frank Williams confirmed that Honda tried to get Mansell sacked from the team so Piquet would get undisputed No.1 status within Williams.

            Honda did then also face complaints of deliberately trying to hurt Williams in 1988 when they pulled out of their contract with Williams at the last minute that year to compromise the design of the FW12. In those circumstances, the perception that Honda would be prepared to sabotage drivers is one that is considered plausible.

  2. I’m really curious as to why george Russell isn’t on that list given he obliterated Kubica in qualifying yet he appears in 18th place. Can’t be due to a lack of F1 experience as Lando appears in 15th….

  3. Montoya?

    1. Exactly, Montoya held the fastest lap ever. At Monza was only beaten last year by these new cars

      1. 7 pole positions in 2002 against one of the greatest car / driver combo’s ever (Schumacher / F2002).

        Most of them were draw-droppingly good laps, none better than Silverstone.

        1. @aussierod

          Yes I agree, JPM is my all time favourite. However, in 2002, as Ross Brawn admitted in his book, Ferrari knew their advantage lay on Sundays, hence they carried more fuel during qualifying. The Williams was rapid over one lap, but they always struggled on Sundays, in many cases the cars ultimate performance was not achieved due to team and driver errors.

          Having said all this, those swashbuckling drives from JPM will always be cherished. Never seen another driver throw the car around like that…but I’m surely biased. haha

          1. @jaymenon10

            2002 was still low fuel qualifying over an hour session I believe. The changes to the format/running order and driving with your starting race fuel came into force in 2003. In 2002 the Williams was an exceptionally good car with the best engine on the grid.

            The only thing stopping JPM challenging for the title was the fact every 5 laps the car would either break down or the engine would detonate itself in the races.

          2. In 2002, cars didn’t qualify with race fuel. It was still the old 12 lap, 1 hour qualifying session.

            Maybe you’re thinking of 2004?

    2. I’m amazed that Montoya is not on that list (that’s why it should not be taken too seriously). He is the fastest guy that I’ve seen on saturdays. Sadly it wasn’t that good on sundays, but JP was fast.

      1. @luigismen I don’t think Montoya was that fast. He got badly beaten by Räikkönen, who isn’t really the fastest driver over one lap. The same can be said about Ralf Schumacher, who was usually no match for Trulli when they were teammates at Toyota.

  4. Once they put all things into quantitative mode, it’s just became less interesting. It’s trivial stuff anyway.

    Because I personally think in the end, it’s about combination of things, like Schumacher on F2004, Kimi Raikkonen on McLaren MP4-20, or Fernando Alonso on Renault R26 being their top performance level.

    1. In the MP4-20 Kimi was untouchable, in a class of one, and team mate Montoya was left far behind.

  5. I wonder how much Kubica’s 2019 season harmed his ranking? I’m impressed he still ranks so high considering his relatively short career and the gap to Russell last season.

  6. Am I right in saying all of Hamiltons F1 team mates are on that list?

    1. Yeah, but his time also considers his gap to this teammates.
      So, he is already as high as he can be.

      1. That doesnt make sense. If he had slower team mates then the relative gap between them would be much bigger thus pushing him further up this list.

        1. So, this means the gap was not so big.

    2. Indeed they are: Alonso, Kovalainen, Button, Rosberg & Bottas. Does that mean he’s dragged them all up, or he just had a lot of fast team mates?

      1. My boy Lewis has made his teammates better. Who says he’s the worst teammate? LOL

      2. I think it shows he’s never had number 1 status to be honest otherwise he’d easily be ahead of Schumacher. Historically its quite rare for teams to not have a outright number 1 who would get preferential treatment and hence skew the performance comparison. Schumacher, Vettel, Verstappen, Alonso, Senna all clearly have had number 1 status and preferential treatment over their teammates for some seasons. That’s not to say they wouldn’t have still beat them but the margins would be less and this is not factored in the ranking either.

        1. What is Nr1 status? Only if there is limited equipment, the faster driver gets it. The same with the logic of applyin team orders. Every team has to make a choice at that time. No team is making its own driver slow on purpose.

    3. He had super fast teammates… alonso, heiki, rosberg, button, Bottas…

  7. It all just sounds like a waste of money and resources to me. Another thing F1 has come up with that nobody asked for (crazy qualifying formats, DRS, tyre management driving). F1 shouldn’t dictate who’s best, let the fans, commentators & pundits decide for themselves, it kinda feels like F1 is shutting down the mother of all f1 related conversations about which driver is best.

    1. Brawn and Smedley made an analysis on who the fastest is. But you still prefer an analysis from some random fanboy on internet.

  8. Both Smedley and Brawn didn’t say anything about how they weighed on drivers age and if drivers weight contributed to the mathematical model.

    It seems all parameters set by human anyway, so it just a spreadsheet with formula, nothing near so called machine learning.

  9. He’ll be back on Kovalainen’s christmas card list for sure.

    It just shows how pointless looking at something like this quantitatively is, really. You can be sure that Trulli is so high because he beat Alonso, then Kov is so high because he beat Trulli, and so on and so on.

    All it exposes is the fallacy of big data models without taking other points into consideration.

    1. Trulli is so high because he beat all his teammates except Kovalainen, who he faced was after his prime.

  10. Still doesnt make sence. Brawn has always been a poletician first, sportsman second and I don’t respect his methodology. By his own words, George Russell who is 26-0 in qualifying against teammates should be number one.

    1. Especially when Russel Beat Kubica, who clearly beat Heidfeld in the past etc.

      1. Was Kubica the same Kubica that beat Heidfeld?

    2. Or does his method only take into account drivers have have driven for at least 2 to 3 teams? No because there is Lando Norris with a worse record against 2 teammates than Russell. Or are drivers in top 4 cars only considered? Really bizarre. I wonder how much Brawn got paid for this crap.

      1. kpcart It is not about the number of times a driver beats his teammate, it is about the gaps to his teammate as well as the quality of the teammate and what their record is compared to all the teammates they have had. GR may be 26-0 against his teammates, but if those teammates aren’t good qualifiers themselves or haven’t done that well against whatever teammates they’ve had, then 26-0 doesn’t mean much.

        1. That is what i thought otherwise the order would be very different. I think they used a minimal seasons of 3 i think and that is why Russels isn’t on this list YET.

    3. thanks forpointing that out. I was going to say exactly the same

    4. Kpcart I guess because Russell’s teammates are ranked low enough that his gap to them isnt high enough to get on this list. So if Kubica is rated as say 1.2 seconds behind Senna, and Russell 0.6 seconds ahead of Kubica, he would still be 0.6 seconds behind Senna and not on this list.

      However, given that there’s no mention of age, injury or form being factored into the equation I’ve no idea why they’d rate Kubica so low when he was clearly very fast in his early career.

      1. @wsrgo You’re right, the gaps should really be displayed in percentages to make it consistent across all circuits. But percentages dont mean as much to fans which is why they chose to display it as a time difference, which must be against some normalised lap time. To make it easy, if you take a lap time of 1 minute 40 (100 seconds), then a gap of 1% would be the same as 1 second. But I dont know time what they used as a baseline – I assume this is stated somewhere in the study but I haven’t seen it in the articles.

  11. Iskandar Mazlan
    20th August 2020, 12:24

    Rosberg beat Michael hands down. Lewis trash Rosberg. Ok Michael was 40 then.

    1. All three sentences are wrong and he was 43 by the time he retired.

      1. @fasterpod Maybe somewhat wrong but the data suggests the link is true: Micheal Nico Lewis. That’s a very short and reliable link unlike Lewis to say Senna. Lewis did beat Nico over their many years. Nico did beat Michael in 2010: https://www.racefans.net/statistics/f1-2010-statistics/#averagegridposition
        and in 2011: https://www.racefans.net/statistics/2011-f1-statistics/2011-f1-statistics-qualifying/
        and in 2012: https://www.racefans.net/statistics/2012-f1-statistics/qualifying-data/

        So according to this link Nico is faster than Michael and Lewis is faster than them both. So what other driver links say otherwise?

        1. @ivan-vinitskyy Rosberg had other teammates in his early career at Williams – Webber,Wurz and Nakajima. He probably lost some rating their because he wasn’t as fast in his early career, losing out to Webber for example by 0.124 seconds on average in qualifying (according to a reddit thread I just found).

        2. @ivan-vinitskyy‘ Rosberg’s previous teammates made him look slower as ‘@keithedin‘ said.
          You wouldn’t expect prime Schumacher to be slower than Webber.
          For Wurz, and Webber, they could do a Schumacher-Massa-Fisichella-Wurz comparison. Or, Schumacher-Massa-Raikkonen-Coulthard-Webber. We don’t know the paths they followed, right?
          And Smedley said that took the ages into account, too.
          I think that’s why 2010-2012 didn’t hurt Schumacher’s time so much and didn’t help Rosberg so much, as well.

    2. Rosberg was far closer (and better at times) to Ham than many people give him credit for.

      Rosbergs most dominant Merc performances made Ham look very very average.

      1. Rosberg beat Hamilton 11-8 in qualifying 2014.

  12. I’m going to put a few names up that did not make the list who in my opinion were just as “fast” not a full list.
    Ronnie Peterson.
    Jim Clark.
    Jochen Rindt.
    Mario Andretti.
    Alberto Ascari.
    Gilles Villenueve.
    Different cars different everything these men had skills way above there most of their compatriots.

    1. @johnrkh the study goes from 1983 onwards, so that’s why those names are not there.

      Not that they’d have put them in a logical order anyway, but that’s the reason why they are not mentioned.

    2. Jeffrey Powell
      20th August 2020, 16:46

      The problem here is that the word ‘ever’ has been used in the write up even though the stats were taken from 1983 onwards. There is no doubt that Jim Clark and Jackie Stewart would be in an ‘ever’ top ten list ,I’d put Jim on top with Senna who if he was still with us would agree. Jochen Rindt was enormously quick but erratic, I was a great fan but did suffer lots of disappointments. If this was fastest f2 driver ever jochen would be No 1.

    3. Ambrogio Isgro
      20th August 2020, 18:39

      When they will go back further, Fangio too will be on the list

  13. Ok then, why was Heikki ousted and sooo unemployable driving in the era of “qualifying gets you results”- F1?
    You’re telling me that keeping the 8th fastest driver in history in a field of 26 was not a wise decision for a team?

    And yes, Racefans did have a laugh at it and promptly put this in the AWS-bin with the rest of the rubbish they spew.

  14. “…because this is a very objective exercise in trying to answer a subjective question…”

    This is an excellent summary of what’s been done here. It’s a great talking point, but we really shouldn’t get wound up about it being flawed. It’s a fun attempt at the impossible. I really like it.

    There’s a whole website (Search “F1Metrics”) where someone has tried to do this with overall driver performance, he/she’s completely unapologetic from the outset that the algorithm has been manipulated over its various versions to account for outliers and bizarre outcomes. But “it is what it is”. An attempt to come up with an objective ranking, which is just quite fun to read.

  15. I’ve always trusted my eyes more than I trust stats, and the biggest omission for me is Hakkinen.

    And without naming names, assuming equal equipment and opportunity in all cases might have given a false picture.

    1. Indeed. Button faster than Mika in qualifying?!

      1. No Button was just on par with very fast teammates, who happened to have 9 world titles between them all. Sure Mika was quick but his teammates were hardly renowned qualifiers either.

    2. Newey rocket helped prop him up.

  16. Iskandar Mazlan
    20th August 2020, 12:31

    Who else in senna n michael era in top 20 list. Mostly from hamilton era. rosberg alonso vettel kimi all wdc.

  17. If F1 really wants to treat statistics seriously and show those AWS graphics make some sense, studies like this will not help to alter people’s perception.

  18. TLDR. Like S…

  19. A question: what are those gaps based on? Is it a mean (or some other summarised central tendency) of all qualifying lap times? That’s weird, because all 20 drivers on the grid haven’t raced on the same circuit, so naturally a 0.1 second gap at the Red Bull Ring shouldn’t be equated to a 0.1 second gap at Spa (Red Bull Ring takes about 60 percent of the time to lap compared to Spa, so if we’re considering gaps maybe a 0.1 second gap at Red Bull Ring should be equated to a 0.16 second gap at Spa).

  20. Ahh.. transitive comparison ranking, so simple and so perfect… until there is a cycle.

  21. “So Driver A’s quicker than Driver B, Driver B goes to another team and he’s quicker than Driver C. So you can say that A is quicker than C because A beat B in his own team and B’s beaten C in another team.”

    While it is nice to have more background info, this just still does not make much sense. Or rather, it is just one way of looking at data, nothing else. Also, this classic argument of ranking drivers eventually can lead to some strange results, like driver being faster than himself, if this A-B-C-D-E chain is long and complicated enough.

    1. Driver is always faster/slower than himself because driver performance keeps changing and/or varying.
      For example Hill beat Villeneuve, Villeneuve beat Frentzen and Frentzen beat Hill.
      We can find the outliers with error correction mechanisms and some simple facts like the fact a driver is far more likely to underperform in relation to teammate than to suddenly become twice the driver he was yesterday.

  22. “I don’t think they’re laughing at it, I think it’s caused some friendly debate,” said Ross Brawn

    Laugh was all I did. And I think I’m not alone.

  23. what a pointless thing to do, really

  24. Would be way more endearing (and entertaining) if it was just the heads of the sport, ie Brawn, Smedley, maybe Carey, getting wasted and publishing drunken bs lists on official F1 outlets if you ask me

  25. Actually the output measured in seconds (gap) is maybe a bit more confusing then outputting something like an ELO score, would create less eyerolling as there are still too much factors in (even fuel, team’s driver preference).

    Comparison to teammate although is one of the most obvious and widespread, fans do it, teams do it, it has relevancy, so that’s a really dominant factor in real life too.

    Prost maybe was not the sharpest qualifiers, but I think if he really would have been 0.5s off of his otherwise really good teammates, then he likely would have worked very hard, so it’s theoretical, and the starting conditions are very diverse per driver. (Senna’s extraordinary performance definitely distorted his placement on this list, as it’s heavily based on teammate comparison anyway). This high gap for Prost not feels too much credible.

    While Lando’s placement is another thing that felt strange, especially considering that no other driver is in the top20 with experience of 20-30 GPs. I think it’s because Verstappen is a really fast qualifier, and Sainz was not far off in their Toro Rosso stint (but was less lucky at reces and car reliability), so that’s the root of Sainz’s good placement, and Norris was quite good compared to Sainz, especially at his 1st season, but he was not that lucky at races and reliability.

    This is why I don’t like the 5s time penalties too much now. Can those penalties be appealed and the race results reinstated? Because after that penalty given competitors around the penalised driver are driving according to that penalty, and using strategy to be inside or outside of that 5 seconds window, so it’s almost impossible to revoke a penalty like that, time penalties burn their marks on the race’s dynamic. I would like to see a much more consistent, although computer aided strong hinting of stewards, and then penalties like that : “give the position back, in one lap” or something like that (based on telemetry and GPS data of same events to decide the time loss or gain of a cut and improper return to the track, because these drivers are so good, that they deliberately can act at some cases as if the cut was unavoidable, and those decisions and judgements are very marginal to the naked eye). That would be more honest, and less destructive to the race’s dynamic. It would be very hard to erase the effect of those 5sec penalties if post race appealing and revoking them would be common.

    1. … telemetry and GPS data of similar events …

  26. This kind of research is what happens when people are ideal and has the same accuracy as the precise weight of 3 fairys eating some cheese.

    1. I meant idle

  27. OK, I have run quick micro-analysis on Kovalainen. In 112 qualifying attempts he has a record of 56/56 when it comes to beating his teammates (Fisichella, Hamilton, Trulli, Chandhok, Petrov, Grosjean) penalties excluded. On average, the driver from the other side of the garage qualified 0.28 grid positions ahead of Heikki. Just saying…

    1. Jose Lopes da Silva
      20th August 2020, 14:12

      Could you publish that?

      1. Well, I have already deleted the worksheet (it could be re-created). But anyway, where would I publish it?!

      2. So if I understand the study correctly, then the reason why Kov has placed as high as he has is that for the 56 times he lost to his teammate in qualifying, the gap wasn’t huge. Eg, Hamilton beat Kovalainen way more often than Kov beat LH, but if Kovalainen was at least close to a driver whose numbers show that he is impressive, then Kovalainen’s closeness to LH even when being beaten by him, bodes well for Kovalainen’s standing in this study. The few times Kovalainen beat LH would also weigh things towards Kov in getting him higher up in the ratings, in part because of LH’s impressive record. So to me .28 positions ahead of Kov as an average for his 6 teammates is not huge, and shows why he was placed as high up as he was.

  28. Jose Lopes da Silva
    20th August 2020, 14:14

    “I’ve always trusted my eyes more than I trust stats”: the kind of worldview that makes people say Covid-19 is a hoax.

    The study is a interesting one and I’ll be waiting for the next developed version.

    1. Actually, being human is being intelligent enough to know when to trust statistics and when to act on gut feel ad tacit knowledge. Clearly Covid is not one of the latter, but many things actually benefit from an intelligent combination of both.

    2. Face, meet palm.

      @john-h covered it perfectly, so I won’t add anything else.

  29. I do kind of get their explanation for not factoring in fuel loads – basically it seemed not worth the effort to correct it since the average difference was small. But weren’t there outliers to that average? Drivers who due to team tactics or driver preference consistently qualified on 2-3 laps less fuel than their teammates, which could make a difference of a tenth or more on their rating. That would make the difference between a driver being 6th or 19th on the table for example.

    I also think driver form is a big influencer in these results, and age/experience could at least be factored in to fine tune the algorithm. Most drivers aren’t as quick in their first couple of years, and after peaking for some years, then tail off later in their careers. And in some cases where drivers had breaks in their career for whatever reason, they are clearly not performing the same if looked at independently. I wonder where Kimi would feature if you only took his results up to 2009, or Kubica pre-injury, or Shumacher pre-comeback (which would have knock-on effects to Rosberg and his teammates)?

    In the end it’s all academic though – the output you get is dependent on the subjective inputs you start with. But I’m sure the teams have better models internally for this kind of analysis than what AWS produced.

  30. The big question I still haven’t seen anyone answer is: All the drivers are given a time delta to Senna, but what is this time related to? Is it a lap of Spa or a lap of the Red Bull ring? 3 tenths at Spa is a lot more respectable to 3 tenths at Austria!

    1. It’s a average over all the tracks used for comparison. It’s not normed to an average length track. They don’t say that. So you can think of it as bigger for spa and less for Austria. It doesn’t really affect the comparative value of the result to me.

      The notable thing about the result is how the top ten is so tightly packed .3-.4s after senna. It kind of confirms the popular mythology of him as being greater than great. And it shows just how small the difference is between being a lefend and an also ran as far as single lap pace goes.

    2. @burden93 It is not about a lap at any one track, it is about a relative gap. Senna was found to be the fastest in quali based in part on the quality of the teammates he had and how much he beat them by, and how they did in quali compared to all their teammates etc etc. Ie. if a driver only ever dominated drivers that themselves have never shown much pace in quali, then that dominating driver wouldn’t be rated as highly as a driver who bested teammates that had very good quali records and therefore were harder to beat.

      So the tenth of a second MS is behind AS is, imho, because on average MS was beating lesser drivers in quali than AS had to. And how it is determined that his teammates were lesser qualifiers is through how they themselves did not just against MS but all other teammates they have had in their tenure in F1. So if MS had WDC level teammates and still beat them by the same time gaps, then he likely would have been first on the list. That he was beating lesser teammates who themselves may not have been all that impressive in quali against their other teammates, shows in the ratings, and this in spite of the much longer run at domination that MS enjoyed that Senna did not. When Senna was at Mac with the most dominant cars, he had a WDC as a teammate. Senna’s poles were much harder fought and won than MS’s were.

  31. Instead of trying to get the competition to speak for itself on the track, we have this absolute nonsense.
    Great way to make F1 a sport.

    1. I’m quite sure the competition is still speaking for itself on the track. I’m sure they’re capable of having some people do this study while F1 carries on racing. And the study is exactly about competition.

      1. Sure absolutely. Sorry about the observations to the contrary.

  32. This is a lot of fun from Smedley and Brawn…..the sort of thing you agree to do in the pub over a beer with a twinkle in your eye! It has certainly had the desired effect.

    1. Kind of, but it’s not a good advert for those involved in ML (i.e. me).

  33. I can’t believe that they are digging-in on this being a reliable, robust analysis, and saying all of these things with a straight face. Really making me lose a lot of respect for Brawn, to be honest.

    When they ran these numbers and ended up with Kovalainen in position 8, and no Montoya or Hakkinen in the top 20 (to name just the most obvious issues) that should have set off some very loud alarm bells. Their reaction to these results should have been “damn! Guess we need to go back to the drawing board”.

    I don’t buy their explanation about the qualifying with race fuel era. Since they have the data of what the fuel difference actually was (apart presumably from some outliers where people retired from the race before their first stop), this should be easy enough to actually put into the model.

    Then there’s the fact that they don’t appear to have accounted for the usual trajectory for drivers’ careers, as others have stated, where they peak for a few years, and then go into decline. This seems to have boosted a few drivers’ rankings, where they were at their peak, driving against others who were in decline and on their way out of the sport.

    Have they accounted for how long drivers were with given teams? You would expect drivers to take a while to acclimatise to a new team for example, and it may hamper their performance initially. For example, when Ricciardo moved to Renault and at first his qualifying vs. Nico was fairly even, but by the end Ric was beating Nico consistently. How about being with a team for a while, and developing the car to your driving style at the expense of your team mate?

    This whole exercise has just served to lower even further my already dim view of these F1 AWS analyses and “insights”. It’s all just total nonsense.

    1. @harrydymond

      For example, when Ricciardo moved to Renault and at first his qualifying vs. Nico was fairly even, but by the end Ric was beating Nico consistently.

      That’s factually wrong. Ricciardo led Hulkenberg 7-1 the first eight races, but the end result was 14-7. Ricciardo basically took no time to settle in at Renault.

      1. Clearly I remembered it wrong! Thanks for the correction.

  34. If you look at differences between teammates appearing on the list (rather than as an absolute metric) it does make sense based on my memories of the races and tends to correct some narratives, Eg about trulli.

    Except for one thing: was Prost really half a second off Senna in the same car? I remember senna would occasionally drop a whole second on the field but if an experienced guy was .5s down on a teammate nowadays he would be a laughing stock.

    Point on Montoya. I think his comparisons fail him. Ralf and Button are not on even the list. Also, his last year with McLaren wasn’t great. His legend is built on some spectacular races in the Williams not on a career of consistent one lap speed.

    1. Button is in the list at no. 17. I admire him but I don’t consider that he was a great in qualifying.

  35. If anyone else had done this exercise, the results would have been scrunched up into a ball and binned. The attached names don’t make the ‘methodology’ any better: A beat B who beat C (who beat D) so A is faster than C (or D). It’s used all the time on internet forums as a comparative metric, but we all know it’s deeply flawed, obliterating far too many factors. (and even allowing for their logic, Schumacher > Rosberg > HAM already exposes its flaws if Schumacher is placed higher than HAM). The key to the best drivers isn’t ‘raw speed’ anyway, it’s adaptation to variabilities to maximize speed. And those variables are constantly changing. Alonso adapted his driving technique to the Renault’s rear downforce and the capacity of the tyres to withstand fairly brutal treatment, gaining enough cornering speed to win two championships. Sure his skill and courage add to his overall quickness anywhere. But it’s that adaptation that made the difference – a technique that wasn’t possible when he switched to McLaren in 2007. Which Alonso was ‘faster’?

    1. @david-br I take your point but they have admitted that they only used quali performances as those were the most apples to apples comparisons to make. And whereas sure a fan will throw out for example DR beat SV in 2014 quali therefore DR > SV, that is obviously moreso an emotional claim rather than one such as this study that goes through all that each driver has done in all their quali sessions combined against the various drivers that have been their teammates.

      You said the key to the ‘best drivers’ isn’t raw speed anyway, but this study is not meant to find the best drivers but it is claiming to have found the fastest drivers.

      1. @robbie Ultimately best = fastest, given we’re talking usually about who completes a race distance quicker on average. In terms of one lap (qualifying) speed, the comparison of team mates in the same year makes some sense, but that’s about it. Trying to compare drivers with different team mates across different cars, years and eras just makes no sense at all. It’s the same kind of algorithmic dross that’s causing no end of problems in other areas.

        1. @david-br Fair comment. I again take your point on the face of it but I’m just trying to appreciate that this study has taken what are the misgivings from you et al, and made adjustments for that. For sure you are right that when we try to talk about who the greats are or the goat, the topic always turns to the difficulty in comparing eras. Yet Smedley and coworkers are doing this and I doubt they think it is dross, which is why I’m trying to make sense of it rather than just dismissing it.

          But anyway, this study is likely not perfect but has once again caused debate and conversation. I do wonder if without the presence of Kovalainen and Trulli there, it would be more palatable and make more sense. I wonder too what the response would have been around here, or how different it would have been had LH been at the top. I bet there would have been a lot of positive remarks about the great and accurate study;)

          1. @robbie Well, I have to agree, probably fewer complaints at any rate if LH was at the top ;P
            There have been a lot of serious issues surrounding private tech consultancy firms using algorithmic studies to advise the UK government lately, including disastrously on Covid-19 and education. I’ve no idea if the people involved consider the results dross or not in private. But they definitely sell their findings as more important than the accumulated experience of specialists in any given area.

          2. @david-br I don’t know about that – there are a lot of people on this site who complain about anything which involves Hamilton, and I am willing to bet those people would complain that it was rigged, biased or otherwise try to tear the study apart.

            Judging by the way that the comments sections usually go in the “star performers” articles, we’d probably also get a number of Verstappen fans complaining that Verstappen should be ahead of Hamilton.

          3. @anon you may be right, who knows, perhaps Verstappen even deserves to be on top as the fastest ever? But how would we know? If he retired tomorrow for example? My point is that there’s no real science involved here, the methodology would be ditched from a peer-reviewed science journal as fundamentally flawed, but the use of ‘algorithms’ gives a gloss that give it a market value (to the gullible).

          4. @david-br Just curious though if there is any difference with this study in that it is not trying to predict anything. So whereas perhaps predictability from algorithms is not necessarily accurate at then guiding a government on what to do about covid for example, this study is not about prediction but just about comparing all drivers and how they have done against all their teammates in qualifying.

          5. @robbie In this case, but the idea is that the same or a similar methodology was and can be used to decide which drivers to sign based on career performance thus far, which would be predictive (or claim to be).

  36. Maybe Smedley can indulge in this rubbish, but there’s better things for Brawn to be doing than this with his time.
    If you put Senna in a Mercedes alongside Hamilton, I highly doubt he would be 0.275 seconds faster, so why this quantitative gap? Max wouldn’t be 0.28 seconds slower than Senna either in a 2020 car. Senna gained massive time on his rivals through stabbing at the throttle and in an era the driver could make much more of a difference.
    Then you have Norris up there. It just shows how rubbish the data is and the assumptions they are making, which the ‘explanations’ only go on to highlight. Get on with fixing DRS or something Ross.

    1. @john-h +10
      Brawn clearly has too much spare time. Which he really shouldn’t have at present.

  37. Smedley and Brawn, LOL. So, this was never a serious exercise.

  38. I’m working in similar analysis in order to evaluate the fastest. From my preliminary results SENNA and CLARK are the fastest followed by SCHUMACHER. The analysis tends to maximize performance of the latest drivers and that is due to career moments (some drivers in end of career and are compared to very young drivers just joining F1). I´m trying to figure out a way to compensate for that.
    I will add driver direct and indirect gap comparisons at Analyticsf1 in instagram (currently I have some racing charts)

  39. Max will be well ahead of Hamilton by the time he finishes his career. Whether he gets 3 or 9 titles won’t really matter.

    Don’t forget that Verstappen basically learned motor racing in F1 as a raw 17 and 18 year old. Yet he’s neck and neck with Hamilton.

    Max is probably another 5 years from his peak as a driver.

    1. David Bondo, Verstappen really isn’t that inexperienced any more given that he’s into his sixth season in the sport and has got more than 100 starts under his belt – he’s already on course to enter the list of the top 50 most experienced drivers in the entire history of the sport in the course of this season.

      Drivers do not keep getting better and better without limit – in fact, the way that Max’s career has developed is pretty much par for the course in terms of driver progression, which is that most drivers usually tend to show the most growth in their career in the first four years, and then tend to reach a plateau after that point.

      I’m not saying there is no room for development, but realistically any growth is likely to be incremental as he’s into a phase of his career where any additional experience is going to be offering far more limited returns than he would have had a couple of years ago. With the level of experience and time in the sport he has already built up, realistically Verstappen will have likely learned the majority of the skills he will ever learn in motorsport by now and will likely already be operating at close to his peak performance.

  40. The devil is in the details, and my educated guess is that it lies in the weights that were manually ascribed to some variables in the model. They are not mentioned here, but the official article from F1 states something in the lines that some weights were used to take into account for instance when a driver is out for a couple of seasons and then comes back. My guess is that there are more artifacts like these. What they need to do is to publish more details on the data (for instance which features were engineered besides the team mate diff), and on the model (at least which external weights were introduced and the sensitivity of the model to variations in those). What they should do is to make the dataset public and let the thousands of fans that are data scientists, analysts, ML experts and whatnot have some fun with it.

  41. “We look at two team mates, exactly the same day, same situation, same opportunity,” Brawn explains. “And we set a time increment between those two team mates and you build that up over time to see how that’s averaging.

    Exactly like I said and all the issues of this not working properly going along with it.

  42. I have been doing a similar analysis using the comparison between qualis. My comparison involves the gaps in % of lap times and Senna and Clark are quicker by some margin. My model, and possibly the AWS, have a tendency of giving a higher rank to most recent drivers, because many new drivers in their early 20s are compared to great drivers that are not in their prime anymore (Schumacher come back makes Nico look amazing…). I´m trying to figure out how to compensate for that. I will be posting the comparisons in my site and Instagram when ready.

    1. I’m always suspicious of the not in their prime variable. Are there any data to show that drivers get slower with age per se. To the contrary you can look at senna at 34, Prost, Schumacher in 2006, mansell, even Hamilton and I don’t see any evidence that they get significantly slower such that you can make this a generalized assumption.

      1. @dmw the general impression seems to be that driver performance tends to start to drop off when a driver reaches their late 30s.

        With the examples you cite, Senna was, at 34 by 1994, not quite old enough for his age to begin having an appreciable impact on his performance. Furthermore, you have to bear in mind the age distribution of the grid as a whole in that period – in the early 1990s, Senna was slightly below the average age of the grid in his era (the average age of drivers on the grid from 1990 to 1993 was between 34-36 years old).

        Mansell certainly was on the older side, but it could be said that, to some extent, his performances were perhaps flattered slightly by the benefits of driving for Williams in the early 1990s, when they had an utterly dominant car (especially in 1992, when there were some races where the gap to the next non-Williams car was sometimes measured in seconds rather than tenths of a second). It is also perhaps notable that quite a few numerical assessments suggest that, in a number of ways, Mansell is probably a bit overrated by fans – particularly in the Anglophone community, perhaps reflecting the “Mansell-mania” of the early 1990s.

        In the case of Schumacher, when he retired at the end of the 2006 season, he was at the age of 37 and the suggestion is that he was probably retired just before he might have begun to show a few signs of age-related decline – had he continued into the 2007 and 2008 seasons, it’s possible that it might have just begun to impact on his performances.

        Hamilton, at the age of 35, is probably also slightly too young for age effects to be that significant – it’s probably going to be a few more years yet before he might show signs of a dip in performance due to age effects.

        1. I believe there is a drop of performance on the late 30s as well. Not only in racing, but many sports. Senna ans Schumi in the early days destroyed their teammates in qualifying and these gaps reduced later in their careers (although with different teammates). In 2006, Massa qualified in front of Schumacher 4 times in 2006 what I find it surprising. I believe the age effect is more determinant on a flying lap, therefore the driver can still be the best on Sundays.

          The issue is how to compensate to age differences on the model…As soon as I have some driver comparisons I will post some charts..

  43. I’m not at all so sure about Senna and Schumacher above Hamilton.
    I think standards have advanced, and the current generation of drivers is measured to the tenth of a second, corner by corner in a way they never were even before 2010.
    During lockdown I watched some old 1980’s and 1990’s career history of Senna, Schumacher et al.
    Viewed in todays light, the drivers around them looked amateurish.

    The discussion of tyre management was at best rudimentary back then, the whole stuck in 1 gear and still doing well (both Senna and Schumacher had their gearbox moments) smells of competition who were weak.

    Standards move on, and you can only compare within their own time. But I do think
    people would be surprised if it were possible to see them in the same car.

  44. That’s a very long winded way of saying “garbage in, garbage out”.

  45. Where is Webber? He often outqualified Vettel as well as all his other teams mates…

  46. So many salty comments! It’s a computer model it can and will be refined but the general story should remain.

Comments are closed.