No, it's all the same in that regard - a ladybug will have a far higher surface area to volume/mass, and that affects terminal velocity.

Ladybug might have 10 square millimeters and weigh .05 grams, 200 square millimeters per gram

Elephant might have 15 square meters and weigh 5000 kilograms. 15 million square millimeters and 5 million grams, so 3 square millimeters per gram

But the elephant in the room (slaps knee) is momentum.

Let's say, hypothetically, we shove a ladybug and an elephant off a 125m cliff and pretend they both have a ridiculously high terminal velocity. That's enough for them to reach 50 meters per second or 180kph. .05 gram ladybug's momentum would be an infinitessimally small 2.5 kg·m/s, meanwhile the elephant is at 250000 kg·m/s, and the elephant explodes.

The thing that makes the ladybug survive the fall (ridiculously low mass relative to surface area) is the same thing that would make a ladybug freeze in minutes if you tossed it in a freezer. Conversely, elephant wouldn't really be bothered by a couple minutes in a freezer.

It's that rapid change in internal body temperature that stresses smaller fish out, dumping them in water that is much colder or warmer than them