The Rust Programming Language

This is an automated archive made by the Lemmit Bot.

The original was posted on /r/rust by /u/venturepulse on 2025-04-25 21:32:41+00:00.

While working on my web research, I ended up writing a small function to make newline characters consistent: either Unix (\n) or DOS (\r\n) style.

I noticed existing crates like newline-converter don't use SIMD. Mine does, through memchr, so I figured I'd publish it as its own crate: newline_normalizer.

Rust has been super helpful for me thanks to the amazing community and tools out there. I thought it’s time to start giving back a bit.

This crate is just a small piece, but it’ll eventually fit into a bigger text normalization toolbox I'm putting together. This toolbox would primarily help data scientists working in natural language processing and web text research fields.