Can you give the Sagemaker instance a security group, and then set the LakeFS ec2 instance security group to allow inbound from the sagemaker security group? then at least you don't have to rely on the sagemaker IP.

Thanks for the response. This was my first thought and attempt. The SageMaker instances live in a AWS managed vpc, and are not directly accessible. The SG attached to the network interfaces used by the instances is whitelisted in the lakefs security group, but I cannot curl the server via this method. I'll work through the issue and respond back if I find a solution.

I think we're not too far behind you in what you're trying to do. Eventually we will have SageMaker instances running that will need to pull data from lakefs. The last resort would be to export the repo branch from lakefs to s3, and have sage maker access the data on s3 directly. I've been experimenting lately with the export feature, for such cases like this that come up.