Axioms

To say that division by zero is undefined is mathematically correct and not just an arbitrary computer limitation. There is no proper answer to the question "What number, when multiplied by zero, gives the non-zero product X?", at least not in any system that would uphold basic idioms such as the product of a number and its reciprocal being equal to one. "Infinity" times zero is not equal to any particular finite number X, so that isn't a solution. Depending on the particular forms of the equations which gave rise to the infinity and the zero (or infinitesimal) the product could be another infinity or any real number; it depends on how you phrase the question. $\lim_{x \to 0+} x ln \frac{1}{x} = 0$, but $\lim_{x \to 0+} x \frac{1}{x} = 1$. In both cases you're multiplying an infinitesimal ($\lim_{x \to 0+} x$) by an infinity ($\lim{x \to 0+} ln \frac{1}{x}$ or $\lim{x \to 0+} \frac{1}{x}$, respectively) but the specific form of the equation changes the result.

You can also construct a quantum field theory for such a universe, it also would work just fine.

Physics tries to model measurements.

So for example you measure a planet that is orbiting, make an equation, and see if tomorrow the equation and the position are the same (within a certain range of precision).

Before Galileo saw that Jupiter had satellites, they had perfectly fine equations that predicted where everything would be in the sky. The problem arose because new data could not fit the model.

You can absolutely model an orbit of a planet using 3 dimensions, or you can model it in an higher space with an equation of a lower degree. Both work. We can't really know which is "exact" if both work.

You can keep adding dimensions and make equations that work, but we don't really know what the "truth" is.