You are correct. For the example of regular languages, we have Kleene algebras, which are special cases of *-semirings. Similar algebras exist for the rest of the Chomsky hierarchy.
Before going up the hierarchy, I would recommend checking out what we can do with semirings alone. Two great papers on the topic are "Fun with Semirings", Dolan 2013 and "A Very General Method of Computing Shortest Paths", O'Connor 2011. Don't be fooled by the titles; they both involve surprise guest appearances from regular expressions.