First public MRP in Spain correctly projects right-wing coalition would fall short of the majority line
This year, YouGov became the first pollster to ever use Multilevel Regression and Post-stratification (MRP) to cover a general election in Spain. We are delighted that the format has once again proved to be a success.
Incumbent Spanish prime minister Pedro Sánchez called the snap election after his party suffered huge defeats in this May’s Spanish local elections.
Our model showed a narrow PP lead, and a hung parliament – an outcome which has come to pass. As with the 2017 UK general election, our prediction flew in the face of much of the rest of the national polling industry, who foresaw a right-wing combined majority for PP and Vox, driven by a substantial polling lead for the centre-right PP.
Our final call six days ahead of voting day (Spanish election laws prohibit publication of polling results any closer to the date) gave a median prediction of 32% of the vote for PP, 28% for PSOE, 14% for Vox and 14% for Sumar. This translated to a median estimate of 133 seats for PP, 113 for PSOE, 42 for Vox and 34 for Sumar.
This has proved very close to the actual outcome, which saw PP gain 33% of the popular vote, PSOE 32%, and Vox and Sumar both on 12%. In terms of seats, PP have won 136 seats, PSOE 122 and Sumar 31. Only the Vox result (33 seats) came outside of our predicted range.
Our predictions for the smaller nationalist parties within Spain's most highly political charged autonomous communities also proved correct – including our expectation that left-wing EH Bildu would overtake their centre-right EAJ-PNV for the first time in the Basque country.
Final days of the race saw PSOE making gains after the legal polling publication period ended
Between our first and second published models for the race, there was a clear trend towards PSOE and away from Vox and Sumar in terms of seats won, and we can now reveal that our final internal call conducted on 18 July – which we weren’t legally allowed to publish due to Spanish electoral law – saw that trend continue.
This later model projected 118 seats for PSOE (an increase of 5 on our model from the day before), 133 for PP (no change), 39 for Vox (a decrease of three), and 32 for Sumar (a decrease of two). This model compares extremely well to the actual results, representing an underestimation of just three on the part of PP and four for PSOE, while overestimating Vox and Sumar by six and one respectively.
One of Sanchez’s key aims in calling the early vote was undoubtedly to disrupt his opposition’s planning and wrong foot them before they could build more momentum. As we can see from the results themselves, and the fact that PP were largely static across all three of our models, he was certainly successful in this regard. His main rival, Alberto Núñez Feijóo of PP, is highly unlikely now to be able to put together a parliamentary majority and take the prime ministership.
Our model saw particular success within key battleground constituencies
One of the great strengths of MRP as a method to predict elections is its ability to correctly capture and estimate local-level variation in party support, and many of the constituency-level results projected by our model were particularly strong. For example, our projected result in both Madrid and Valencia proved extremely accurate:
A key component to any election in Spain is the issue of independence in two of the country’s most politically-charged autonomous communities – Catalonia and the Basque country. This snap election was no different, with Vox in particular keen to try and weaponise the issue against PSOE over negotiations and coalition arrangements with nationalist parties.
There was also an intriguing story developing in the Basque country, with the prospect that EH Bildu – once banned by the Spanish Supreme Court over alleged associations with the political wing of the former Basque terrorist group Euskadi Ta Askatasuna – could draw level with their right-wing nationalist counterparts EAJ-PNV in terms of seats won within Paìs Vasco itself. If they then managed to hold onto their seat in next door Navarra, this would put them above EAJ-PNV in terms of total seats for the very first time.
Our model projected the outcome within the Paìs Vasco autonomous community to within just one seat difference.
Meanwhile, PSOE would go on to win no fewer than 19 seats across Catalonia on Sunday, up 7 from November 2019. In Barcelona itself, our forecast was again close to the eventual result. While we did not quite capture the scale of the PSOE advance, we estimated that they would add to their existing total in Spain’s second largest city as the left-wing Catalonian nationalist ERC party fell back. We also correctly expected PP to advance strongly in Barcelona, rising from two seats to five.
YouGov’s first Spanish election compares very well to the rest of Spain’s polling industry
The majority of pollsters before the election were calling the contest in favour of a PP+Vox majority – Spain’s “great swing to the right” dominated much of the news coverage. However, a small band out of the total eighteen pollsters modelling the election bucked this trend in projecting a hung parliament, with PP struggling to break significantly beyond PSOE in terms of vote share. YouGov were among that latter group, along with other organisations such as GESOP and 40dB.
As well as successfully calling the result, we were unrivalled in terms of the accuracy of our seat forecasts. From our final published model (17 July), our mean error of just 4.8 seats across all parties projected leads the Spanish polling industry in terms of accuracy.
Our closest competitors in terms of accuracy of seat projections were 40dB (~5.3 mean error), Cluster17 (~5.5), and Sondaxe (~5.7).
Again, this strong accuracy in terms of seats is testament to MRP as a method, which allowed us to be far more granular in our estimations of how individual seats within constituencies would be allocated than traditional methods allow.
In terms of vote share, our mean squared error across all parties estimated was 2.7 (meaning all parties were accurately captured within an average error of about 1.6 points). This placed us well toward the top for accuracy on this metric.
Our unpublished 18 July model performed even better. In terms of vote share, this later model had a mean squared error of just 1.9 across all parties projected. This unpublished model had a mean error of just 2.83 seats across all parties projected.
Where we will look to improve
The outcome of the vote makes it very likely further elections will be held later this year. With 176 seats required for a majority, the 169 seats won secured by PP and Vox means they will struggle to put together any kind of workable coalition. Even with the addition of regionalist parties from the Canary Islands and Navarra, they would remain short of being able to command support of parliament. More fanciful coalitions with nationalist parties – the very kind that Vox wish to see disbanded – would be required to take the Spanish right over the line.
Ahead of that likely coming election, we will be looking at where our model underperformed so that we can make even more accurate predictions.
Firstly, this was an election in which the entire polling industry lowballed the incumbent PSOE. Rather than obtaining the expected 27-29% of the vote, Sanchez’s party reached 32%.
Our miss on PSOE was four points, lying outside what we would typically accept as the sort of statistical error we might expect with measuring party support. Understanding where the model failed to correctly capture PSOE support will be one of our core focuses for the coming months.
Similarly, though we were not too far away in terms of estimating Vox’s overall support, they were the only party whose eventual seat total came outside our predicted intervals. Our two point overstating of their national support translated to a nine-seat overinflation on our part.
The same did not happen with Sumar, however, who we also overestimated by two points in terms of vote share but were very accurate on in terms of seat wins. Unpicking this puzzle will be another main part of our work moving forward.
Overall, though, we can be extremely pleased with how our first foray into projecting Spanish elections – and our first attempt at using MRP to do so outside of the UK and the USA – has panned out.
