https://lakefs.io/ logo
#dev
Title
# dev
y

Yoni Augarten

02/06/2023, 3:55 AM
Hey, I'm starting lakeFS using docker-compose, with postgres as the backing database. I'm using
replicas: 3
to scale up - and on the first run I'm getting the following error on one of the lakeFS containers, and it fails to start:
Copy code
setup failed: ERROR: relation \"kv_49\" already exists (SQLSTATE 42P07)
Seems like a race condition. Any recommendations on how to deal with it?
b

Barak Amar

02/06/2023, 7:31 AM
can you share the docker-compose?
y

Yoni Augarten

02/06/2023, 7:49 AM
Copy code
version: "3.9"
services:
  postgres:
    image: postgres:13
    environment:
      POSTGRES_USER: lakefs
      POSTGRES_PASSWORD: lakefs
      POSTGRES_DB: lakefs
    volumes:
      - postgres-db-volume:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "lakefs"]
      interval: 5s
      retries: 5
    restart: always
  
  lakefs:
    image: treeverse/lakefs:0.92.0
    deploy:
      replicas: 3
    ports:
      - "8001"
    environment:
      - AWS_ACCESS_KEY_ID
      - AWS_SECRET_ACCESS_KEY
      - LAKEFS_BLOCKSTORE_TYPE=s3
      - LAKEFS_DATABASE_TYPE=postgres
      - LAKEFS_DATABASE_POSTGRES_CONNECTION_STRING=<postgresql://lakefs:lakefs@postgres:5432/lakefs?sslmode=disable>
      - LAKEFS_AUTH_ENCRYPT_SECRET_KEY=a123456
      - LAKEFS_LOGGING_LEVEL=INFO
      - LAKEFS_LOGGING_AUDIT_LOG_LEVEL=INFO
      - LAKEFS_LISTEN_ADDRESS=0.0.0.0:8001
    depends_on:
      postgres:
        condition: service_healthy
    volumes:
      - ~/.aws:/home/lakefs/.aws:ro

  nginx:
    image: nginx:latest
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - lakefs
    ports:
      - "8000:8000"
volumes:
  postgres-db-volume:
b

Barak Amar

02/06/2023, 8:17 AM
no lock when creating partitions
we use "create if not exists" so I assume the warning is there, but it should work
y

Yoni Augarten

02/06/2023, 8:43 AM
But it crashed the container- it's an error, not a warning
b

Barak Amar

02/06/2023, 9:25 AM
Copy code
lfs-postgres-1  | 2023-02-06 07:51:49.353 UTC [124] ERROR:  relation "kv_91" already exists
lfs-postgres-1  | 2023-02-06 07:51:49.353 UTC [124] STATEMENT:  CREATE TABLE IF NOT EXISTS"kv_91" PARTITION OF "kv" FOR VALUES WITH (MODULUS 100,REMAINDER 91);
Copy code
cdff8ade65ba   postgres:13                     "docker-entrypoint.s…"   2 hours ago   Up 2 hours (healthy)   5432/tcp                            lfs-postgres-1
strange the statement say "CREATE IF NOT EXISTS" I expected postgres to ignore the error if the table is already there
still wip - can verify that it solves the error you are getting
Updated - let me know if you have an issue or you need me to open one
y

Yoni Augarten

02/06/2023, 11:52 AM
Reviewed! Thank you
119 Views