https://lakefs.io/ logo
#help
Title
# help
u

唐治喜

01/07/2024, 1:27 PM
Hi there, can lakefs use mariadb? And how to setup which the docker-compose.yaml file? version: '3.9' services: minio: image: treeverse/lakefs:1.6 container_name: lakefs ports: - "18000:8000" - "18080:8080" environment: - LAKEFS_BLOCKSTORE_TYPE=s3 - LAKEFS_BACKEND_TYPE=s3 - LAKEFS_S3_ENDPOINT=http://192.168.120.22:19000 - LAKEFS_S3_ACCESS_KEY_ID=oxJU - LAKEFS_S3_SECRET_ACCESS_KEY=sZlYWvhF5 - LAKEFS_METADATA_STORE_TYPE=mysql - LAKEFS_DATABASE_TYPE=mysql - LAKEFS_DATABASE_HOST=192.168.2.22 - LAKEFS_DATABASE_PORT=3306 - LAKEFS_DATABASE_NAME=lakefs_db - LAKEFS_DATABASE_USERNAME=qft - LAKEFS_DATABASE_PASSWORD=oB2jJq5oWUIIxkS - LAKEFS_AUTH_ENCRYPT_SECRET_KEY=hoz2W4tSmI5N volumes: - /lakefs_data:/lakefs/data
g

Guy Hardonag

01/07/2024, 2:09 PM
Hi @唐治喜 We currently do not support mysql as the database (we do support postgresql) In a previous conversation in #dev, there was a suggestion to add it as a contribution https://lakefs.slack.com/archives/C01APUNTSGH/p1702144339703259
👍 1
u

唐治喜

01/09/2024, 1:29 AM
Thanks,it seems solved with the postgres,but i can't connect with minio, and no tables showed up in the database lakefs_db in postgres;how can I solve these?Thanks a lot! here are logs from postgress: 2024-01-07 141629.948 UTC [1] LOG: database system is ready to accept connections 2024-01-07 152503.972 UTC [946] ERROR: column c.relhasoids does not exist at character 190 2024-01-07 152503.972 UTC [946] STATEMENT: SELECT c.oid, n.nspname AS schemaname, c.relname AS tablename, c.relacl, pg_get_userbyid(c.relowner) AS tableowner, obj_description(c.oid) AS description, c.relkind, ci.relname As cluster, c.relhasoids AS hasoids, c.relhasindex AS hasindexes, c.relhasrules AS hasrules, t.spcname AS tablespace, c.reloptions AS param, c.relhastriggers AS hastriggers, c.relpersistence AS unlogged, ft.ftoptions, fs.srvname, c.reltuples, ((SELECT count(*) FROM pg_inherits WHERE inhparent = c.oid) > 0) AS inhtable, i2.nspname AS inhschemaname, i2.relname AS inhtablename FROM pg_class c LEFT JOIN pg_namespace n ON n.oid = c.relnamespace LEFT JOIN pg_tablespace t ON t.oid = c.reltablespace LEFT JOIN (pg_inherits i INNER JOIN pg_class c2 ON i.inhparent = c2.oid LEFT JOIN pg_namespace n2 ON n2.oid = c2.relnamespace) i2 ON i2.inhrelid = c.oid LEFT JOIN pg_index ind ON(ind.indrelid = c.oid) and (ind.indisclustered = 't') LEFT JOIN pg_class ci ON ci.oid = ind.indexrelid LEFT JOIN pg_foreign_table ft ON ft.ftrelid = c.oid LEFT JOIN pg_foreign_server fs ON ft.ftserver = fs.oid WHERE ((c.relkind = 'r'::"char") OR (c.relkind = 'f'::"char")) AND n.nspname = 'public' ORDER BY schemaname, tablename here is the lakefs yaml for docker-compose version: '3.9' services: lakefs: image: treeverse/lakefs:1.6 container_name: lakefs ports: - "8000:8000" - "8080:8080" environment: LAKEFS_BLOCKSTORE_TYPE: s3 LAKEFS_BACKEND_TYPE: s3 LAKEFS_BLOCKSTORE_S3_FORCE_PATH_STYLE: true LAKEFS_BLOCKSTORE_S3_ENDPOINT: http://192.168.122.2:9000 LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID: oxGtJOXRNowp LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY: sZlj8D5uTFcaeHY76jz LAKEFS_DATABASE_TYPE: postgres LAKEFS_DATABASE_POSTGRES_CONNECTION_STRING: postgresql://lkadmpost:Oi0V3Ja@192.168.2.22:5432/lakefs_db LAKECTL_CREDENTIALS_ACCESS_KEY_ID: AuYSmIES LAKECTL_CREDENTIALS_SECRET_ACCESS_KEY: hoz2W4HYHjtSmI5NEKEY LAKEFS_AUTH_ENCRYPT_SECRET_KEY: YWvhFk323rOu5ou volumes: - /lakefs_data:/lakefs/data
a

Ariel Shaqed (Scolnicov)

01/09/2024, 6:10 AM
Hi @唐治喜, That sounds like a Postgres error, perhaps it was not fully set up on your machine. I see this error message on this StackOverflow answer, it may be relevant: https://stackoverflow.com/questions/58461178/how-to-fix-error-column-c-relhasoids-does-not-exist-in-postgres
u

唐治喜

01/22/2024, 7:23 AM
thanks! all resolved!but there are errors in the browser console panel while i viewing the objects,what do these matter? Failed to load resource: the server responded with a status of 400 (Bad Request) index-bxs-Ce9N.js:1 HEAD http://10.10.1.22:18000/api/v1/repositories/qfordivdata/refs/main/objects?path=lakes.parquet 404 (Not Found) N @ index-bxs-Ce9N.js:1 head @ index-bxs-Ce9N.js:1 La @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 Wd @ vendor-g6fb1NZy.js:40 m3 @ vendor-g6fb1NZy.js:40 WL @ vendor-g6fb1NZy.js:40 Rl @ vendor-g6fb1NZy.js:38 lZ @ vendor-g6fb1NZy.js:40 T2 @ vendor-g6fb1NZy.js:40 yP @ vendor-g6fb1NZy.js:40 E @ vendor-g6fb1NZy.js:25 R @ vendor-g6fb1NZy.js:25 index-bxs-Ce9N.js:1 HEAD http://10.10.1.22:18000/api/v1/repositories/qfordivdata/refs/main/objects?path=README.md 404 (Not Found) N @ index-bxs-Ce9N.js:1 head @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 Wd @ vendor-g6fb1NZy.js:40 m3 @ vendor-g6fb1NZy.js:40 WL @ vendor-g6fb1NZy.js:40 Rl @ vendor-g6fb1NZy.js:38 lZ @ vendor-g6fb1NZy.js:40 T2 @ vendor-g6fb1NZy.js:40 yP @ vendor-g6fb1NZy.js:40 E @ vendor-g6fb1NZy.js:25 R @ vendor-g6fb1NZy.js:25 index-bxs-Ce9N.js:1 GET http://10.10.1.22:18000/api/v1/repositories/qfordivdata/settings/gc_rules 404 (Not Found) N @ index-bxs-Ce9N.js:1 getGCPolicy @ index-bxs-Ce9N.js:1 isCompleted @ index-bxs-Ce9N.js:1 E @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 Wd @ vendor-g6fb1NZy.js:40 m3 @ vendor-g6fb1NZy.js:40 WL @ vendor-g6fb1NZy.js:40 Rl @ vendor-g6fb1NZy.js:38 lZ @ vendor-g6fb1NZy.js:40 T2 @ vendor-g6fb1NZy.js:40 yP @ vendor-g6fb1NZy.js:40 E @ vendor-g6fb1NZy.js:25 R @ vendor-g6fb1NZy.js:25 index-bxs-Ce9N.js:1 GET http://10.10.1.22:18000/api/v1/repositories/qfordivdata/settings/gc_rules 404 (Not Found) N @ index-bxs-Ce9N.js:1 getGCPolicy @ index-bxs-Ce9N.js:1 isCompleted @ index-bxs-Ce9N.js:1 E @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 Wd @ vendor-g6fb1NZy.js:40 m3 @ vendor-g6fb1NZy.js:40 (anonymous) @ vendor-g6fb1NZy.js:40 E @ vendor-g6fb1NZy.js:25 R @ vendor-g6fb1NZy.js:25 index-bxs-Ce9N.js:1 HEAD http://10.10.1.22:18000/api/v1/repositories/qfordivdata/refs/main/objects?path=README.md 404 (Not Found) N @ index-bxs-Ce9N.js:1 head @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 Wd @ vendor-g6fb1NZy.js:40 m3 @ vendor-g6fb1NZy.js:40 WL @ vendor-g6fb1NZy.js:40 Rl @ vendor-g6fb1NZy.js:38 lZ @ vendor-g6fb1NZy.js:40 T2 @ vendor-g6fb1NZy.js:40 yP @ vendor-g6fb1NZy.js:40 E @ vendor-g6fb1NZy.js:25 R @ vendor-g6fb1NZy.js:25 3index-bxs-Ce9N.js:1 Uncaught (in promise) TypeError: Cannot read properties of null (reading 'id') at E (index-bxs-Ce9N.js147794) at index-bxs-Ce9N.js147968 at Array.forEach (<anonymous>) at index-bxs-Ce9N.js147942 at Wd (vendor-g6fb1NZy.js4024283) at m3 (vendor-g6fb1NZy.js4042409) at vendor-g6fb1NZy.js4040726 at E (vendor-g6fb1NZy.js251576) at MessagePort.R (vendor-g6fb1NZy.js251942) E @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 Wd @ vendor-g6fb1NZy.js:40 m3 @ vendor-g6fb1NZy.js:40 (anonymous) @ vendor-g6fb1NZy.js:40 E @ vendor-g6fb1NZy.js:25 R @ vendor-g6fb1NZy.js:25 index-bxs-Ce9N.js:1 HEAD http://10.10.1.22:18000/api/v1/repositories/qfordivdata/refs/main/objects?path=lakes.parquet 404 (Not Found) N @ index-bxs-Ce9N.js:1 head @ index-bxs-Ce9N.js:1 La @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 Wd @ vendor-g6fb1NZy.js:40 m3 @ vendor-g6fb1NZy.js:40 (anonymous) @ vendor-g6fb1NZy.js:40 E @ vendor-g6fb1NZy.js:25 R @ vendor-g6fb1NZy.js:25 index-bxs-Ce9N.js:1 GET http://10.10.1.22:18000/api/v1/repositories/qfordivdata/settings/gc_rules 404 (Not Found) N @ index-bxs-Ce9N.js:1 getGCPolicy @ index-bxs-Ce9N.js:1 isCompleted @ index-bxs-Ce9N.js:1 E @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 Wd @ vendor-g6fb1NZy.js:40 m3 @ vendor-g6fb1NZy.js:40 (anonymous) @ vendor-g6fb1NZy.js:40 E @ vendor-g6fb1NZy.js:25 R @ vendor-g6fb1NZy.js:25 index-bxs-Ce9N.js:1 GET http://10.10.1.22:18000/api/v1/repositories/qfordivdata/settings/gc_rules 404 (Not Found) N @ index-bxs-Ce9N.js:1 getGCPolicy @ index-bxs-Ce9N.js:1 isCompleted @ index-bxs-Ce9N.js:1 E @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 Wd @ vendor-g6fb1NZy.js:40 m3 @ vendor-g6fb1NZy.js:40 (anonymous) @ vendor-g6fb1NZy.js:40 E @ vendor-g6fb1NZy.js:25 R @ vendor-g6fb1NZy.js:25 index-bxs-Ce9N.js:1 GET http://10.10.1.22:18000/api/v1/repositories/qfordivdata/settings/gc_rules 404 (Not Found) N @ index-bxs-Ce9N.js:1 getGCPolicy @ index-bxs-Ce9N.js:1 isCompleted @ index-bxs-Ce9N.js:1 E @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 (anonymous) @ index-bxs-Ce9N.js:1 Wd @ vendor-g6fb1NZy.js:40 m3 @ vendor-g6fb1NZy.js:40 (anonymous) @ vendor-g6fb1NZy.js:40 E @ vendor-g6fb1NZy.js:25 R @ vendor-g6fb1NZy.js:25
b

Barak Amar

01/22/2024, 9:53 AM
Can you set
blockstore.s3.disable_pre_signed: true
in your lakeFS configuration? it seems your UI fails to access the storage. I assume the minio storage.
u

唐治喜

01/22/2024, 12:33 PM
Thanks! I added this item to my docker-compose.yaml, but errors still. Whereas I created annother repository which "Add sample data, hooks, and configuration" checked, and then click to view it, no error shows up in the browser control panel, and then I clicked the directory "data", and the error occurred again, it seems lakefs always check the "README.md“ file ? and the " lakes.parquet" file?
b

Barak Amar

01/22/2024, 12:44 PM
can you share the docker-compose (mark secrets if found)? want to verify if any settings is missing
u

唐治喜

01/22/2024, 12:49 PM
version: '3.9' # 使用最新的Docker Compose版本以利用新特性 services: lakefs: image: treeverse/lakefs:1.6 # 使用最新稳定版镜像(或指定特定版本) container_name: lakefs ports: - "8000:8000" # 根据实际需求决定是否暴露9001端口,用于HTTPS访问时需要 - "8080:8080" # 根据实际需求决定是否暴露9001端口,用于HTTPS访问时需要 environment: LAKEFS_BLOCKSTORE_TYPE: s3 LAKEFS_BACKEND_TYPE: s3 # 使用S3兼容存储后端 LAKEFS_BLOCKSTORE_S3_FORCE_PATH_STYLE: true LAKEFS_BLOCKSTORE_S3_DISABLE_PRE_SIGNED: true LAKEFS_BLOCKSTORE_S3_ENDPOINT: LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID: oxGRwp LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY: sZlj8DTFcaeHYHv76jz LAKEFS_DATABASE_TYPE: postgres LAKEFS_DATABASE_POSTGRES_CONNECTION_STRING: # 如果使用PostgreSQL作为数据库(可选) #LAKECTL_CREDENTIALS_ACCESS_KEY_ID: AuLtSmIES #LAKECTL_CREDENTIALS_SECRET_ACCESS_KEY: hoz2ouY5FcaeHY5NEKEY LAKEFS_AUTH_ENCRYPT_SECRET_KEY: YWvh23rOu5ou LAKECTL_CREDENTIALS_ACCESS_KEY_ID: AKIAYII5FQ LAKECTL_CREDENTIALS_SECRET_ACCESS_KEY: KuvjIsKUJ/hdC volumes: - /lakefs_data:/lakefs/data # 确保宿主机目录存在并有合适的权限
this is the example repository, no error,
then click the data directory above, errors occured
I added those two empty files to the former repository, and the console is clear
b

Barak Amar

01/22/2024, 1:53 PM
The UI will try to get README.md on every path and draw preview of the readme. So 404 is expected.
In the console screenshot I didn't see any error related to lakefs file
u

唐治喜

01/22/2024, 2:11 PM
thanks, these can be useful notice for testing lakefs with minio.
b

Barak Amar

01/22/2024, 2:42 PM
The UI will fetch the README.md on all underlying storage implementations.
3 Views