School leaders' union the NAHT said teachers and students deserve praise for their hard work but warned the nature of the tests can put children with special needs at a disadvantage.
Since 2022, all pupils in England are expected to take a standardised multiplication test at the end of Year 4.
New figures from the Department for Education show children receiving SEND support in Hampshire scored an average of 16 out of 25 this year, while those with no extra needs scored 21.6.
Nationally, pupils with SEND support scored 17.1, while those without had an average of 21.9.
Sarah Hannafin, head of policy at the NAHT, said the union maintains the test is unnecessary, but accepts "it is here to stay".
"Given that, it’s important that changes are made to improve the accessibility of the test," she said.
"We have consistently raised concerns that its timed nature and focus on memorisation puts some children, including pupils with certain types of special educational needs, at a disadvantage.
"Although the Government says it will work with the Standards and Testing Agency on 'minimising unintended impacts' it is disappointing that it appears to have ruled out flexibility on the timed nature of the exercise.
"The truth is that some pupils may be able to come up with the right answers, but may need a bit longer than their peers, and they should not be penalised for that."
The data also shows how students classified as 'disadvantaged' are lagging behind their peers.
Disadvantaged pupils are those known to be eligible for free school meals, have been in the care of the local authority or have recently left local authority care.
In Hampshire, they scored 18.2, less than the 21.3 from those not disadvantaged.
Nationally, these scores were 19.3 and 21.7 respectively.
Pepe Di’Iasio, general secretary of the Association of School and College Leaders, said: "It is testament to the hard work of schools and teachers that attainment in the multiplication tables check has continued to improve for pupils from both disadvantaged and non-disadvantaged backgrounds.
"However, there remains a gap between these groups of children which is also the case across other indicators, and is rooted in the fact that disadvantaged children face huge challenges in their lives.
"The recently announced child poverty strategy is a welcome step in the right direction, but as a society, we have to do more to back schools with greater investment for targeted support for these young people."
Nationally, boys scored an average of 21.2, more than the 20.7 for the girls.
Both groups increased their score by 0.3 points compared to the previous year.
Boys in Hampshire scored an average of 20.9, compared to 20.3 for the girls.
A DfE spokesperson said: "Basic multiplication is an essential life skill, and our revitalised curriculum will make sure every child leaves school with rock solid foundations, from science to reading to maths.
"High and rising standards are at the heart of this Government’s mission to break down barriers to opportunity so every child can achieve and thrive.
"That’s why from September 2026, teachers in reception to Year 2 will receive new training to strengthen children’s early maths skills, and our expanded Maths Champions programme will support up to 1,800 nurseries and 60,000 children, helping disadvantaged pupils make up to six months of extra progress."