https://lakefs.io/ logo
Join Slack
Powered by
I'm trying to use Hadoop FS but hitting `Failed to...
# help
r
I'm trying to use Hadoop FS but hitting 
Failed to get lakeFS blockstore type
. Details in notebook - could someone point where I'm going wrong please? I'm using Everything Bagel with MinIO for storage https://gist.github.com/rmoff/c3d6553aec11b569d8f1e1761b7182e5
n
Do you have the right permissions for this request? GetStorageConfig requires ReadConfigAction permissions
r
permissions on the storage? or lakeFS?
n
lakefs permissions
r
I'm using a user that's in the 
Admins
group
n
Is there anyway to get some more information on the error? What status code was returned from lakefs?
r
I don't see any message in the lakeFS log
n
What's the lakeFS log level?
r
it was 
INFO
. Even with it set to 
DEBUG
I don't see anything from lakeFS, just the notebook
Copy code
jupyter-notebook  | 23/05/31 16:01:03 WARN FileSystem: Failed to initialize fileystem <lakefs://test/main/users>: java.io.IOException: Failed to get lakeFS blockstore type
ah, if I turn up Spark logging I get this:
Copy code
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FileSystem: Starting: Acquiring creator semaphore for <lakefs://test/main/users>
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FileSystem: Acquiring creator semaphore for <lakefs://test/main/users>: duration 0:00.001s
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FileSystem: Starting: Creating FS <lakefs://test/main/users>
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FileSystem: Looking for FS supporting lakefs
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FileSystem: looking for configuration option fs.lakefs.impl
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FileSystem: Filesystem lakefs defined in configuration option
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FileSystem: FS for lakefs is class io.lakefs.LakeFSFileSystem
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FsUrlStreamHandlerFactory: Creating handler for protocol http
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FsUrlStreamHandlerFactory: Unknown protocol http, delegating to default implementation
jupyter-notebook  | 23/05/31 16:05:39 WARN FileSystem: Failed to initialize fileystem <lakefs://test/main/users>: java.io.IOException: Failed to get lakeFS blockstore type
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FileSystem: Failed to initialize fileystem
jupyter-notebook  | java.io.IOException: Failed to get lakeFS blockstore type
n
Copy code
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FsUrlStreamHandlerFactory: Creating handler for protocol http
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FsUrlStreamHandlerFactory: Unknown protocol http, delegating to default implementation
This looks suspicious
Can you check if you can curl to the endpoint directly?
r
the lakeFS or minio one?
n
lakeFS
r
I'll need to install curl on the container
it's the same endpoint that I use from the lakeFS client on the same container though, so the network should be good.
n
I suspect the spark configuration is the problem - just wanted to make sure the lakefs endpoint you provided is valid
👍 1
r
Copy code
root@be6ae10d9902:/# curl <http://lakefs:8000/api/v1>
{"message":"invalid API endpoint"}
n
Copy code
curl <http://lakefs:8000/_health>
r
Copy code
# curl <http://lakefs:8000/_health>
alive! c0d5c906-e316-4029-b093-6b6057d079fc
👍🏽 1
(TIL 🙂 )
😄 1
n
So I suspect some thing with the spark configuration is not right. Specifically this might be a direction to investigate:
Copy code
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FsUrlStreamHandlerFactory: Creating handler for protocol http
jupyter-notebook  | 23/05/31 16:05:39 DEBUG FsUrlStreamHandlerFactory: Unknown protocol http, delegating to default implementation
Unfortunately I'm not a spark expert. Maybe @Yoni Augarten or @Ariel Shaqed (Scolnicov) can help?
a
@Yoni Augarten could this be a manifestation of what you fixed with this PR?
r
I've logged https://github.com/treeverse/lakeFS/issues/5997 to track and for visibility beyond the walls of Slack 🙂
70 Views
Open in Slack
PreviousNext
Powered by