Conduit and PostgreSQL

For a while now I’ve been playing around with an event-drive software design (EDA) using conduit for processing of events. For this post the processing can basically be viewed as the following diagram

+-----------+   +------------+   +---------+
|           |   |            |   |         |
| PG source |-->| Processing |-->| PG sink |
|           |   |            |   |         |
+-----------+   +------------+   +---------+
     ^                                |
     |            +------+            |
     |            |      |            |
     |            |  PG  |            |
     +------------|  DB  |<-----------+
                  |      |
                  +------+

I started out looking for Conduit components for PostgreSQL on Hackage but failed to find something fitting so I started looking into writing them myself using postgresql-simple.

The sink wasn’t much of a problem, use await to get an event (a tuple) and write it to the database. My almost complete ignorance of using databases resulted in a first version of the source was rather naive and used busy-waiting. Then I stumbled on PostgreSQL’s support for notifications through the LISTEN and NOTIFY commands. I rather like the result and it seems to work well.1[]

It looks like this


  1. If I’ve missed something crucial I would of course love to hear about it.

Felipe Lessa

I’ve never used LISTEN/NOTIFY myself, so take this with a grain of salt. From my understanding of the docs, you only need to call LISTEN once:

LISTEN registers the current session as a listener on the notification channel named channel. If the current session is already registered as a listener for this notification channel, nothing is done.

Furthermore, there’s a race condition in the first invocation of dbSource: a notification that arrives after the SELECT but before the LISTEN would be lost.

Assuming my understanding is correct, here’s some totally untested code:

Magnus

Thanks Felipe, that’s exactly the kind of comment I was hoping for. I’ll make sure to adjust my code based on your insights.

Leave a comment