Today @Lior Itzhak, @Itai Admi and I were discussing the behavior of the lakeFS hadoop file system when copying into a non-existing directory. I've checked the behavior in both HDFS and S3A (without lakeFS). It turns out that

hadoof fs -cp

doesn't copy to a non-existing path (on neither HDFS nor S3A). What one should do is use

-mkdir -p

to create the path, and then perform the copy.

Neat! Is there some hadoopfs spec test that would have shown us this, or is this another case of "do whatever S3A does"?

Well, this behavior is defined by the hadoop cli, which is one layer above what we are implementing. So it was just making sure that a sequence of commands behaves as expected.