Private Diffie/Helman Tokens

> My question still stands...Why do we need "Let's Ecrypt"?

You need it to protect against active attackers.

Doing a Diffie-Hellman is enough to protect against passive attackers, who can only observe but not modify the traffic. But against an active attacker, who can modify the packets being exchanged, that's not enough; the attacker can replace the "tokens" (which, from what I understand, are what you are calling the "public key" part of the Diffie-Hellman key exchange) with some of their own, and sit in the middle pretending to be the other side of the exchange. To prevent that, you have to somehow validate that these "tokens" actually came from the real other side of the exchange.

That's where systems like Let's Encrypt fit: they validate that the "public key" received from the peer actually came from them, and not from an attacker intercepting and modifying the connection.