In the last few years, a focus in language modelling has been on improving performance through increasing the number of parameters in transformer-based ...