https://lakefs.io/ logo
#help
Title
# help
h

HT

10/12/2023, 8:36 AM
Question around Active connection from lakefs to KV store: We deployed in Azure with Postgres server, It seems that even with an empty lakefs, no repo, lakefs seems to always maintain a number of connection to the databse server near the
LAKEFS_DATABASE_POSTGRES_MAX_OPEN_CONNECTIONS
value. Is this expected ? Why does it maintain all those connections even if there are no request going to the front end of the lakefs server ?
i

Isan Rivkin

10/12/2023, 11:59 AM
Hey @HT ! We’re using pgxpool go package for the connection pool management. • What lakeFS version are you using? • What are your Postgres configs? • Where are you reading seeing the metrics of open connections? in Azure or in lakeFS? ◦ in lakeFS we have prometheus stats published in
/metrics
and the metric
pgxpool_acquired_conns
shows the current open connections to KV. What do you see in there?
h

HT

10/12/2023, 8:18 PM
• Old version: 107. The plan is to update to the release that contain https://github.com/treeverse/lakeFS/issues/6718 • We use the default config provide by Azure Postgres server. Do you have some that you would be interested ? Because there are 412 .... • I am reading from the Azure Metrics: TRAFFIC > Active Connection. Below plot is with 2 lakefs server sharing the same database server. Lakefs config: 50 Max connection and 25 idle connection • I will have a look at prometheus stats
metric:
@Isan Rivkin any insight ?
i

Isan Rivkin

10/15/2023, 2:46 PM
Hey @HT, the max_idle_connection logically is min_connections and that’s why it keeps 25 min connections. I see how that can be super confusing, please treat the max_idle_conns as minimum conns.
👍 1
h

HT

10/15/2023, 8:48 PM
Is there any guideline on how to tune those numbers ? I read this https://docs.lakefs.io/howto/sizing-guide.html but it did not talk about those connection to the database ... How does those connection related to the a S3 list for example ? Or a rclone sync ?
Should I use Idle_conn (aka min) == 1 and max_conn == 100 ?
i

Isan Rivkin

10/16/2023, 6:22 AM
Yes, if you wish min conns to be 1 then use Idle_conn as 1, same for max_conns.
you can read more about relation to storage (s3) and Postgres in here
3 Views