That's ok if we are talking about malware publicly shown in the published source code.. but there's also the possibility of a private source-code patch with malware that it's secretly being applied when building the binaries for distribution. Having clean source code in the repo is not a guarantee that the source code is the same that was used to produce the binaries.

This is why it's important for builds to be reproducible, any third party should be able to build their own binary from clean source code and be able to obtain the exact same binary with the same hash. If the hashes match, then you have a proof of the binary being clean. You have this same problem with every single binary distribution, even the ones that don't include pre-compiled binaries in their repo.