This is a very fair concern.

However, the thing to consider is with these Smart TV's, and generally poor privacy companies and devices, is that they'll always use the lowest hanging fruit where possible. I'd say the amount of data that the device will go through the lengths to collect offline and later upload are quite low.

Most Smart TV's collect data from when people directly use the Smart TV itself (i.e. what apps and content you directly watch and download on it as an example). The vast majority of people who buy TV's like this will just use the built-in Smart TV for watching media and for doing everything on the device, without even giving it a second thought, so the manufacturers basically automatically win and get data on 99% of consumers, without really any effort or work.

For people like us who do care and go through the trouble of circumventing it (i.e. not using the built-in Smart TV at all, and disconnecting the TV entirely from the internet besides for updates occasionally), I'd say that only leaves 2 issues: listening through the mic (I think this is unlikely since voice recordings take up a lot of storage space, as well as bandwith to upload, and like I said in my first comment, TV manufacturers really cheap out on these devices and cut corners wherever possible, and back to my point of the low hanging fruit, but regardless, if you're concerned, then removing the mics or taping them would certainly solve this), and recording or storing what you view through other inputs like your streaming box (again I feel this is unlikely for same reasons as voice recording, it'd take up a lot of space and bandwith, isn't quite practical, and they typically focus on the low hanging fruit).

These are both possible concerns, but I'd say they're realistically extremely unlikely, especially due to how much effort and work it would take to spy on such a small portion of people. The investment and the amount of work and effort needed to do this just doesn't make sense and isn't really justified for them. If you're still concerned, then I'd recommend just putting your TV behind a VPN and putting it into a separate VLAN whenever you do connect or update it (and also just taping/removing the microphone like I said above), or of course if you're extra paranoid, you can always just leave it fully offline and deal without updates. But I personally just don't see it as a major risk or something to worry about.

There definitely needs to be regulations on these devices, its completely unacceptable in its current state. It shouldn't be this hard to just get something like a TV without it spying on you and completely invading your privacy, but I guess that's the world we live in now. :/