You're both right!
But also note that your workflow using rebase on Git satisfies some but not all possible requirements. First, as you explained, you need to handle conflicts. But that's just metadata for a single branch. The commit is still there, it is still accessible by tag or by other branches or by digest. Now you need to delete all tags pointing at that commit, and also rebase all branches dependent on that commit. Some of these will include merges; there is no porcelain in Git for rebase-across-merges, so I think you lose merge metadata.
It gets simpler after that: if you want that data deleted from your storage, that's still not enough. The data goes away only when Git gc's it. Eventually you'll have a GC run on your data, and then it gets deleted... if no commits reference it.