Please don't assume anything, it's not healthy.

Explicitly stating assumptions is necessary for good communication. That's why we do it in research. :)

it depends on the license of that binary

It doesn't, actually. A binary alone, by definition, is not open source as the binary is the product of the source, much like a model is the product of training and refinement processes.

You can't just automatically consider something open source

On this we agree :) which is why saying a model is open source or slapping a license on it doesn't make it open source.

the main point is that you can put closed source license on a model trained from open source data

1. Actually the ability to legally produce closed source material depends heavily on how the data is licensed in that case
2. This is not the main point, at all. This discussion is regarding models that are released under an open source license. My argument is that they cannot be truly open source on their own.