Binary speed is really the least reason to use Gentoo.

There are a lot of thorny issues in package distribution that source builds completely sidestep.

Install-it-yourself plus source updates are a lot to ask, but if you can get the hang of it the benefits are pretty sweet.