https://lakefs.io/ logo
Join Slack
Powered by
Hi, are the examples <https://github.com/treeverse...
# help
f
Hi, are the examples https://github.com/treeverse/charts/blob/master/examples/lakefs/enterprise/values-oidc.yaml still valid as of today? found some differences with https://docs.lakefs.io/reference/security/sso.html aswell trying to deploy with the following and I'm getting YAML parse error on
Copy code
lakefs/templates/fluffy-deployment.yaml: error converting YAML to JSON: yaml: line 85: did not find expected '-' indicator (sadly I can't see the fluffy-deployment.yaml :disappointed: )
this is my cfg just in case
Copy code
fluffy:
            enabled: {enterprise_enabled}
            fullname: {fluffy_fullname}
            image:
                repository: treeverse/fluffy
                tag: '0.2.0'
                pullPolicy: IfNotPresent
                privateRegistry:
                    enabled: true
                    secretName: {unique_service_name}dockercfg
            fluffyConfig: |
                logging:
                    format: "json"
                    level: "INFO"
                installation:
                    fixed_id: fluffy-authenticator
                auth:
                    post_login_redirect_url: /
                    logout_redirect_url: <https://lakefs.services.x.com/oidc/logout>
                    oidc:
                        enabled: true
                        url: <https://x.okta.com>
                        client_id: {okta_clientid}
                        callback_base_url: <https://lakefs.services.x.com>
                        is_default_login: true
                        logout_client_id_query_parameter: client_id
                        logout_endpoint_query_parameters:
                            - returnTo
                            - <https://lakefs.services.x.com/oidc/login>
            secrets:
                create: false
            sso:
                enabled: true
                oidc:
                    enabled: true
                    client_secret: dummyValueEnableSecretLogic
            rbac:
                enabled: true
i
Hey @Florentino Sainz! Thanks for providing the config, it seems that the issue is with YAML / Helm formatting, the usage of 
{}
is not common in value files. I used your config and got an identical error, then I changed the following values and it rendered with no errors. 1. 
enabled: true
2. 
secretName: changed-secret
The command I executed to test: 
helm template -f values.yaml lakefs lakefs/lakefs >  final-dump.yaml
Contents of 
values.yaml
Copy code
fluffy:
            enabled: true
            fullname: {fluffy_fullname}
            image:
                repository: treeverse/fluffy
                tag: '0.2.0'
                pullPolicy: IfNotPresent
                privateRegistry:
                    enabled: true
                    secretName: changed-secret
            fluffyConfig: |
                logging:
                    format: "json"
                    level: "INFO"
                installation:
                    fixed_id: fluffy-authenticator
                auth:
                    post_login_redirect_url: /
                    logout_redirect_url: <https://lakefs.services.x.com/oidc/logout>
                    oidc:
                        enabled: true
                        url: <https://x.okta.com>
                        client_id: {okta_clientid}
                        callback_base_url: <https://lakefs.services.x.com>
                        is_default_login: true
                        logout_client_id_query_parameter: client_id
                        logout_endpoint_query_parameters:
                            - returnTo
                            - <https://lakefs.services.x.com/oidc/login>
            secrets:
                create: false
            sso:
                enabled: true
                oidc:
                    enabled: true
                    client_secret: dummyValueEnableSecretLogic
            rbac:
                enabled: true
f
thanks Isan!, {} is python string placeholders sorry, they do get replaced in my example to real strings. to me they do get generated correctly, but kubernetes rejects the generated, ima try with helm directly to see if it helps debugging, thanks
i
NP! LMK if you need additional help 👍
f
ok issue found, is in the chart
Copy code
nodeSelector:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.affinity }}
      affinity:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.tolerations }}
      tolerations:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.topologySpreadConstraints }}
        topologySpreadConstraints:
          {{- toYaml . | nindent 8 }}
      {{- end }}
https://github.com/treeverse/charts/blob/master/charts/lakefs/templates/fluffy-deployment.yaml
if you compare it with https://github.com/treeverse/charts/blob/master/charts/lakefs/templates/deployment.yaml you can see, topologySpreadContraints is "displaced" a few characters, which breaks the generation
sorry, I thought it was because of fluffy, its just that when adding fluffy I enabled the deployment-fluffy.yaml xP, but its because I have topologySpreadConstraints and nodeSelector set-up in my values.yaml
@Isan Rivkin should I open a ticket?
this is the values which reproduces the issue
Copy code
existingSecret: lakefs-auth-encrypt-secret
      fullnameOverride: lakefs
      serviceAccount:
          create: false
          name: "lakefs"
      replicaCount: 3
      resources: 
          requests:
              cpu: 1800m
              memory: 15Gi
      committedLocalCacheVolume:
        hostPath:
          path: /data/
      topologySpreadConstraints:
        - maxSkew: 1 
          topologyKey: <http://topology.kubernetes.io/zone|topology.kubernetes.io/zone>
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
              matchLabels:
                  app: lakefs
          matchLabelKeys:
              - pod-template-hash
      lakefsConfig: |
          database:
              type: dynamodb
              dynamodb:
                  table_name: ${Token[TOKEN.2193]}
                  aws_region: eu-west-1
          committed:
              local_cache:
                  size_bytes: 10737418240
          blockstore:
              type: s3
              default_namespace_prefix: s3://${Token[TOKEN.1466]}/lakefs/
              s3:
                  region: eu-west-1
                  disable_pre_signed_ui: true
                  pre_signed_expiry: 1h
                  web_identity:
                      session_duration: 1h
                      session_expiry_window: 50m
          gateways:
              s3:
                  region: eu-west-1
          auth:
              oidc:
                  friendly_name_claim_name: name
                  default_initial_groups: []
              ui_config:
                  login_url: /oidc/login
                  logout_url: /oidc/logout
                  login_cookie_names:
                      - internal_auth_session
                      - oidc_auth_session
      fluffy:
          enabled: true
          fullname: lakefs-fluffy
          image:
              repository: treeverse/fluffy
              tag: '0.2.0'
              pullPolicy: IfNotPresent
              privateRegistry:
                  enabled: true
                  secretName: lakefsdockercfg
          fluffyConfig: |
              logging:
                  format: "json"
                  level: "INFO"
              installation:
                  fixed_id: fluffy-authenticator
              auth:
                  post_login_redirect_url: /
                  logout_redirect_url: <https://lakefs.services.whatmocked.com/oidc/logout>
                  oidc:
                      enabled: true
                      url: <https://whatsys.okta.com>
                      client_id: 0oa1udg0csr840fhc0h8
                      callback_base_url: <https://lakefs.services.whatmocked.com>
                      is_default_login: true
                      logout_client_id_query_parameter: client_id
                      logout_endpoint_query_parameters:
                          - returnTo
                          - <https://lakefs.services.whatmocked.com/oidc/login>
          secrets:
              create: false
          sso:
              enabled: true
              oidc:
                  enabled: true
                  client_secret: dummyValueEnableSecretLogic
          rbac:
              enabled: true
helm template -f values.yaml lakefs lakefs/lakefs > final-dump.yaml
i
Nice catch! It’s like that for 3 months, i guess none used topologySpreadConstraints since than or upgraded. Thank you, I’ll create a PR now to fix it 🙏
f
@Isan Rivkin Actually I think topologySpreadConstraints was added because of my request, maybe we are the only ones using it 🙂 ,we didn't have the license yet 3months ago so I didn't test fluffy/wasn't aware about it, I remember the issue was there for deployment.yaml too (which was fixed because I was testing that one, but probably fluffy was missed out)
Anyways, will remove topologySpreadConstraints for now and add it back whenever the fix is done (we are not yet prod-serving so its not so big deal)
i
Created the PR already https://github.com/treeverse/charts/pull/217 will update when merged !
gratitude thank you 2
@Florentino Sainz done new version is out, 
1.0.6
can you please try? 🙂
dont forget to update 
helm repo update
👍 1
f
trying
i
oops, one moment it’s not published yet by the CI so helm repo update will not not work
ok done now 🙂
👍 1
f
worked thanks! now its time to debug fluffy on my end 🙂 but that's another work
i
Thank you for uncovering that bug! LMK if I can help with the rest of fluffy. (BTW Im in the meeting we have today 🙂 )
f
🙂 thanks! i will if I get stuck, but for now its time to fix settings here and there, trying to get something almost working before the meeting.
jumping lakefs 1
@Isan Rivkin for https://github.com/treeverse/charts/blob/master/charts/lakefs/templates/_fluffy.tpl is it possible that {{- if .Values.existingSecret }} - name: LAKEFS_AUTH_ENCRYPT_SECRET_KEY valueFrom: secretKeyRef: name: {{ .Values.existingSecret }} key: {{ .Values.secretKeys.authEncryptSecretKey }} {{- else if and .Values.secrets (.Values.secrets).authEncryptSecretKey }} - name: FLUFFY_AUTH_ENCRYPT_SECRET_KEY valueFrom: secretKeyRef: name: {{ include "lakefs.fullname" . }} key: auth_encrypt_secret_key {{- else }} - name: FLUFFY_AUTH_ENCRYPT_SECRET_KEY value: asdjfhjaskdhuioaweyuiorasdsjbaskcbkj {{- end }} the first one should be FLUFFY_AUTH_ENCRYPT_SECRET_KEY ?
I think its a copy-paste issue from https://github.com/treeverse/charts/blob/master/charts/lakefs/templates/_env.tpl (ExistingSecret was also added under my request I think, secrets when deploying using AWS CDK need to be created ahead of it/outside of CDK, otherwise they are visible in the AWS CDK template xP)
👀 1
i
@Florentino Sainz Again you’re right, on it - PR will update when merged
@Florentino Sainz Fixed please use version 
1.0.7
and thank you! 🙏
👍 1
gratitude thank you 1
f
ty! tested and working, now time to fight with Okta xP but fluffy pod is running
i
👍
installation id: to go the lakefs 
/_health
you should see 
alive! <installation id>
Copy code
# Source: lakefs/templates/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: lakefs
  labels:
    helm.sh/chart: lakefs-1.0.7
    app: lakefs
    app.kubernetes.io/name: lakefs
    app.kubernetes.io/instance: lakefs
    app.kubernetes.io/version: "1.3.1"
    app.kubernetes.io/managed-by: Helm
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      if ($http_x_forwarded_proto = 'http') {
        return 301 https://$host$request_uri;
      }
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/server-snippet: |
      location ~* "^/(metrics|_pprof)" {
          deny all;
          return 403;
        }
spec:
  ingressClassName: internal-alb
  rules:
    - host: "lakefs.services.acme.com"
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: lakefs
                port: 
                  number: 80
          
          - path: /oidc/
            pathType: Prefix
            backend:
              service:
                name: fluffy-sso
                port: 
                  number: 80
          - path: /api/v1/oidc/
            pathType: Prefix
            backend:
              service:
                name: fluffy-sso
                port: 
                  number: 80
          - path: /saml/
            pathType: Prefix
            backend:
              service:
                name: fluffy-sso
                port: 
                  number: 80
          - path: /sso/
            pathType: Prefix
            backend:
              service:
                name: fluffy-sso
                port: 
                  number: 80
          - path: /api/v1/ldap/
            pathType: Prefix
            backend:
              service:
                name: fluffy-sso
                port: 
                  number: 80
Copy code
ingress:
  enabled: true
  ingressClassName: alb
  annotations:
    <http://nginx.ingress.kubernetes.io/configuration-snippet|nginx.ingress.kubernetes.io/configuration-snippet>: |
      if ($http_x_forwarded_proto = 'http') {
        return 301 https://$host$request_uri;
      }
    <http://nginx.ingress.kubernetes.io/force-ssl-redirect|nginx.ingress.kubernetes.io/force-ssl-redirect>: "true"
    <http://nginx.ingress.kubernetes.io/server-snippet|nginx.ingress.kubernetes.io/server-snippet>: |
      location ~* "^/(metrics|_pprof)" {
          deny all;
          return 403;
        }
  hosts:
    - host: <http://lakefs.services.acme.com|lakefs.services.acme.com>
      paths: 
       - /
override api keys (credentials login page)
Copy code
ingress:
  enabled: false
  annotations: {}
    # <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: nginx
  # <http://kubernetes.io/tls-acme|kubernetes.io/tls-acme>: "true"
  ingressClassName: ""
  defaultBackend: {}
  hosts:
    - host: chart-example.local
      paths: []
      # redirect to a different service based on path prefix for advanced use cases only
      # pathsOverrides:
      #   - path: /auth/login
      #     serviceName: <lakefs-root>
      #     servicePort: 80
1. adding extra ingress (internal) using extraManifests 2. fluffy replicaCount:
Copy code
fluffy:
    replicaCount: 3
19 Views
Open in Slack
PreviousNext
Powered by