LWN: Comments on "Varrazzo: Thinking psycopg3"

Varrazzo: Thinking psycopg3

malefic — Mon, 30 Nov 2020 05:08:36 +0000

It really is very simple. Python async functions are just generators, which you can "call" or "iterate" over manually. The key part in blocking_proto.pyx is _iter_coroutine, which is basically all you need to "call" an async function. You don't need an event loop or anything. It's the non-blocking I/O part that `yield`-s to the event loop that makes the code async, once all your sockets are blocking it basically becomes regular blocking code with a weird calling convention.

Seriously?

Wol — Tue, 17 Mar 2020 12:18:39 +0000

Is it just that he doesn't suffer fools gladly?

And it's very noticeable in society generally that as people age they get more cantankerous. I just shrug and say to myself "I hope I don't get like that as I get older".

Cheers,
Wol

Varrazzo: Thinking psycopg3

auc — Mon, 16 Mar 2020 11:10:36 +0000

Looking at blocking_proto.pyx, I'm a bit suprised you need something so small. In the asyncpg source, async/await is all over the place (incl. cython parts). I just don't understand what you're doing there.

Varrazzo: Thinking psycopg3

auc — Mon, 16 Mar 2020 11:05:27 +0000

Yes of course, I did my own loop-per-thread management. Still fails in obscure ways I hadn't the immediate time budget for.

Seriously?

ballombe — Mon, 16 Mar 2020 10:21:14 +0000

I do not know. Language has been toned down but the attacks have gone more vicious.

Varrazzo: Thinking psycopg3

malefic — Sat, 14 Mar 2020 20:13:01 +0000

asyncio is not thread safe, you must use a separate event loop in each thread and never call loop or task methods from another thread (with the exception of the explicit loop.call_soon_threadsafe()).

Varrazzo: Thinking psycopg3

auc — Sat, 14 Mar 2020 17:22:19 +0000

I disagree with your choice of words but honestly I am very disappointed with the way Python has been managed by the core devs ... for the past ten years. We basically had two forks: Python 3 and Asyncio. The later is a fork within the language itself. I guess this probably happened because Python around 2.5 was basically a done thing, and what's more funny than breaking things ?

Varrazzo: Thinking psycopg3

auc — Sat, 14 Mar 2020 17:11:43 +0000

I attempted the sync-facade in front of the async loop. It works until you start doing threads. Then it breaks down in the most obscure ways. In all fairness I only invested like, three full days into this, but the experiment was not too encouraging. Asyncio is such a big mess :/

Still I'll have a look at what you've done, thanks for the link :)

Seriously?

corbet — Sat, 14 Mar 2020 16:05:02 +0000

Do note that the comment you linked to is from 2012... The archives are full of stuff from that source that is not matched by what he posts today. I can actually understand those who feel that we have lost something, he has a way with words that few can match. But we are stronger as a community if we are not constantly attacking each other.

Seriously?

HelloWorld — Sat, 14 Mar 2020 16:01:26 +0000

Yeah… I guess it's OK when Al Viro does it though.

https://lwn.net/Articles/513188/

Seriously?

corbet — Sat, 14 Mar 2020 15:26:23 +0000

Come on now. If you think that async could be better handled in Python, well that's fair game (though a bit off-topic on this article). But calling the work of people who have worked their ass off and given the results away a "turd" really is not. Why are you so attached to kindergarten-level insults?

This account has been the source of rather too many issues like these; moderation bit is now on.

Varrazzo: Thinking psycopg3

jrigg — Sat, 14 Mar 2020 10:57:58 +0000

> I'm glad I don't need to work with that turd a lot.

There seems to be a lot of this around here lately. I don't like Python much, but I just use something else without feeling the need to trash the work of others.

Varrazzo: Thinking psycopg3

malefic — Sat, 14 Mar 2020 01:59:01 +0000

It is entirely possible to have an async-first driver with a small synchronous facade on top. See the edgedb-python driver [1] for how it can be done. Unfortunately, we made a mistake of relying on futures too much in asyncpg. It will need to be refactored to use coroutines instead before this approach can be applied. I'll be happy to help with reviews and guidance if you're willing to take this on!

[1] https://github.com/edgedb/edgedb-python/blob/master/edged...

Varrazzo: Thinking psycopg3

HelloWorld — Sat, 14 Mar 2020 00:09:55 +0000

It's pretty pathetic that in the Python world you still need to worry about whether something is async or not. Every time I read about Python I'm glad I don't need to work with that turd a lot.

Varrazzo: Thinking psycopg3

auc — Fri, 13 Mar 2020 17:25:21 +0000

Don't miss asyncpg (https://github.com/MagicStack/asyncpg) which is an awesome python/postgresql driver. My only regret is that it is tied to the async monad, which makes it basically useless within the non-async world. Maybe this year I will do an attempt at a non-async variant of this ...