It is not a 1-1 ratio. It takes a large amount of the brain just to control all the 'background programs' needed to live. Once you have that complete, you can add mass but the brain doesn't have to grow as much. If someone is 10% bigger, the brain will only need to become a small percentage larger to control that extra mass, nowhere near 10%. So that 10% of our brain that we lost equals us being pygmy sized.