https://lakefs.io/ logo
Join Slack
Powered by
I think dns is ok, and lakefs is work, but I can n...
# help
m
I think dns is ok, and lakefs is work, but I can not use aws cli connect my lakefs S3…
a
Please try to post ASCII (in between triple backquotes) rather than screen images. It is not possible o copy-paste from screenshots. Now specifically, 
<http://s3.local.lakefs.io|s3.local.lakefs.io>
is a DNS record that we (lakefs) created in to connect to your loopback. As such it will work for processes on your machine, but not inside a container.
Inside a container (Docker, K8s, etc.), you should use that platform's facilities to give a DNS record. The above record just resolves to 127.0.0.1, which will be the wrong choice for a container. What to do? For docker-compose, our "everything" docker-compose file https://github.com/treeverse/lakeFS/blob/master/deployments/compose/docker-compose.yml (
deployments/compose/docker-compose.yml
) shows you how to set up everything. AND it gives you almost everything you need to get a small demo environment running on your workstation, including Spark, Trino (Presto), Hive, MinIO, ...
Please let us know if this works better :-)
m
8d1be4b2c73e treeverse/lakefs:latest “/app/wait-for postg…” 8 days ago Up 8 days 0.0.0.0:8000->8000/tcp lakefs_lakefs_1
this is my docker port mapping
I think the local access can touch my docker port now, I can access lakefs’s UI, but can not use aws cli …
a
From your machine it should work. From inside another container it should not.
m
I am not sure why… if my docker’s port can be access now, the local dns why not work…
I have add this to my hosts file
10.2.0.10:8000 can be access now
o I know you mean now…
I run aws cli inside docker, the docker can not use my hosts file
is that right?
so the question should be if we use aws cli tool by docker run mode, how to let the --endpoint-url know the dns name means… : (
a
tl;dr: yes. The gory details: s3.local.lakefs.io is just a DNS record we serve that points at your localhost:
Copy code
ariels@redqueen:~/dev/lakeFS/clients/spark$ dig <http://s3.local.lakefs.io|s3.local.lakefs.io>

; <<>> DiG 9.16.6-Ubuntu <<>> <http://s3.local.lakefs.io|s3.local.lakefs.io>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19438
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;s3.local.lakefs.io.            IN      A

;; ANSWER SECTION:
<http://s3.local.lakefs.io|s3.local.lakefs.io>.     86400   IN      A       127.0.0.1

;; Query time: 168 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Wed Aug 11 14:17:24 IDT 2021
;; MSG SIZE  rcvd: 63
So it works to access a lakefs that listens on port 8000 on your machine, from your machine, but not from a Docker container on that machine. And forwarding the lakeFS port out doesn't help, either. The 
docker-compose.yml
I referenced tells Docker to add a DNS entry for the lakefs server for the relevant containers. Now those containers can access the lakefs server. Hope I managed to make sense.
m
thanks much, think the easy way to solve is not use aws cli tool by doker mode : |
o
@Melfe Bulu yea, networking (both name resolution and routing) makes life a bit more challenging with Docker. Host->container usually works pretty well though. Let us know if run into any further issues or if you have more questions!
m
it work now 🙂
can go bicycel now ^^
🚴🏾 1
🚴🏽 1
lakefs 1
🤘 1
Open in Slack
PreviousNext
Powered by