I also mix'n'match upper and lower receivers with no issue. As long as they are properly manufactured there should be no huge functional issues but there can be cosmetic differences (whether that matters is up to you).
Personally, the rollmark on the lower receiver does matter to me and I will pay more for what I want. There are some rollmarks I wouldn't have (even if properly made) because I won't have "that" name on my custom build. In saner times only $25-50 more would buy a premier name to go along with the rest of the quality parts. And often that extra money does buy a better finish and more importantly a flared magwell. No matter what the quality of the components in the rifle, the rollmark does scream to anyone who sees it the perceived quality of the build.
Also you asked about billet vs. forged receivers and again there is no huge difference. For the same strength, forged is slightly lighter but not more than a few ounces. For the extra money billet can look fantastic, with a great finish and flared magwell, and often has niceties like an integrated trigger guard and bolt hold-open screw that eliminate tapping in roll pins. And most have a tensioning screw to remove any slop between the upper and lower. Without using the tensioning screw, my Quentin Defense US Army Edition (guess why I chose that rollmark!
) billet lower fits and functions fine with four different forged uppers but of course many folks like to match billet to billet.