railroads are far more sensitive to grade and unevenness than roads are.
don't discount the cost of the rails, over miles that really amounts to a large amount
also, all the control points where other traffic needs to get stopped to let the railroad go through cost
then you have the need to engineer specific places where the traffic can be redirected (switches, etc)
what makes railroads efficient to run is that they have very low friction, and as a result you can pull the same load with less power. As a result of having lower power for a given weight (and less friction with the tracks), trains accelerate slower than normal vehicles. due to less friction, they decelerate slower as well. As a result you want to avoid having the train make speed changes as much as possible
if you are going to stop frequently (and don't have a huge volume of passengers that need to fit in each vehicle), then dedicated busways (2 lane roads usable only by buses, with railroad like crossing guards) are actually cheaper, and the buses can leave the busways and use normal streets as well.