A wise person once said, > _"With great scale c...
# dev
a
A wise person once said,
_"With great scale comes big data_"
And we're getting there. I think it's time for pack files like Git uses, so I made up this issue to design how we can do them. Packs serve as snapshots in big data (or cubes if anyone still remembers OLAP): in this case, I would like us to hold many commits together in a single "file". The issue has a sketch, and I think we need to think about this direction. I believe it can help us with: • GC on large repositories with many commits • lakectl blame and other log-by-path operations • Even FindMergeBase when the base is distant. It won't be a short sprint, but I'd like us to think about this as a possible next step for lakeFS scalability.