Does LakeFS support https out of the box OR do we need to stand up reverse proxy like nginx in front of it?

Take a look at the

tls.*

fields of the configuration. This is fine for debugging. However keeping private keys is tricky, and I would strongly recommend decrypting TLS on a well-managed proxy endpoint.