Blindsight by Peter Watts. It's the quintessential modern take on cosmic horror. First there is the "external" horror of a truly alien spaceship that is compared to a "crown of thorns" and "devil's baklava" with its strange inhabitants, and then there is the "internal" horror of musings about the nature and relationship between conciousness and intelligence. Then there is the transhuman main cast, including Jukka Sarasti the vampire and the AI of the ship Theseus. The sidequel Echopraxia is also great, expanding on the concepts and introduces concepts like human hivemind, militarized zombies and Portia-like alien intelligence.

Killing Star by Zebrowski and Pellegrino discusses the Dark Forest hypothesis with various subplots about scientific ambition gone wrong, personal loss, paranoia and religious zeal (some nutjobs cloned Jesus and Buddha, and the clones, after raising their eyebrow in amusement, went "Nah, we're outta here!"). The novel starts with 99.9% of humans and life on Earth wiped out in relativistic bombing, and then it gets worse. The attacking aliens and their technology is well thought out and the way they hunt down the last remnants of humanity is harrowing.