Author: Alok

DevSecOps

Integrating Jenkins login with Keycloak

  • Run Keycloak with docker-compose
version: '3'
services:
  postgres:
      image: postgres:9.6
      volumes:
        - /opt/postgres/:/var/lib/postgresql/data
      environment:
        POSTGRES_DB: keycloak_db
        POSTGRES_USER: keycloak_user
        POSTGRES_PASSWORD: password
  keycloak:
      image: quay.io/keycloak/keycloak:latest
      environment:
        DB_VENDOR: POSTGRES
        DB_ADDR: postgres
        DB_DATABASE: keycloak_db
        DB_USER: keycloak_user
        DB_SCHEMA: public
        DB_PASSWORD: password
        KEYCLOAK_USER: admin
        KEYCLOAK_PASSWORD: admin
      ports:
        - 8080:8080
      depends_on:
        - postgres
  • Open Keycloak panel : http://192.168.0.184:8080 > add realm > Jenkins
  • Add client, Client > client-protocal=openid-connect > Root URL
  • Client > Jenkins-client > Installation > Keycloak OIDC JSON > Download

Note : Take the back up of /var/jenkins_home/config.xml or take a snapshot if it’s vm.

  • Manage Jenkins > Configure system > Global Keycloak Settings > add downloaded json data > Save
  • Manage Jenkins > Configure global security > Securiy Realm > Keycloak Authentication Plugin > Save and logout
  • Create Users in Keycloak realm “jenkins” and login with user(eg. admin1)

More – https://www.keycloak.org/getting-started/getting-started-docker

Custom Daemonset command based on host_ip in kubernetes

Why?
– When we need to add some extra functionally to daemonset based on which worker node it’s running on

apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels:
    app: custom-daemonset
  name: custom-daemonset
spec:
  selector:
    matchLabels:
      app: custom-daemonset
  template:
    metadata:
      labels:
        app: custom-daemonset
    spec:
      containers:
      - command:
        - /bin/bash
        - -c
        - |
          echo "$STARTUP_SCRIPT" > /tmp/STARTUP_SCRIPT.sh
          /bin/bash /tmp/STARTUP_SCRIPT.sh
        env:
        - name: HOST_IP
          valueFrom:
            fieldRef:
              fieldPath: status.hostIP
        - name: STARTUP_SCRIPT
          value: |
            #!/bin/bash
            if [ $HOST_IP == "192.168.0.184" ]; then
              echo "HOST_IP is $HOST_IP"
            else
              echo "HOST_IP does not match $HOST_IP"
            fi
            sleep 600
        image: nginx
        imagePullPolicy: IfNotPresent
        name: custom-daemonset

Ref : https://github.com/kubernetes/kubernetes/issues/24657#issuecomment-577747926

AD integration with linux ssh login and sudo access

ad_join.sh

#!/bin/bash

#check if already joined to domain

if [[ $(realm list) != "" ]]
then
echo "This server is already joined to domain."
realm list | head -n 1
exit
fi

function update_sssd_config() {
    sed -i 's/use_fully_qualified_names = True/use_fully_qualified_names = False/g' /etc/sssd/sssd.conf
    sed -i 's|/home/%u@%d|/home/%u|g' /etc/sssd/sssd.conf
    systemctl restart sssd
}

function restrict_ssh_access_group() {
    if [[ $(cat /etc/ssh/sshd_config | grep -o "updated_by_ad_join") != "updated_by_ad_join" ]]
    then
    echo "###############updated_by_ad_join.sh###############" >> /etc/ssh/sshd_config
    echo "AllowGroups root ssh-access-group" >> /etc/ssh/sshd_config
    systemctl restart sshd
    fi
}

function sudo_access_level_group() {
    if [[ $(cat /etc/sudoers | grep -o "updated_by_ad_join") != "updated_by_ad_join" ]]
    then
    echo "###############updated_by_ad_join.sh###############" >> /etc/sudoers
    echo "Cmnd_Alias SUDO_ACCESS_LEVEL1 = /usr/bin/ls, /usr/bin/cat " >> /etc/sudoers
    echo "Cmnd_Alias SUDO_ACCESS_LEVEL2 = /usr/bin/vi, /usr/bin/nano " >> /etc/sudoers

    echo "%sudo-group-level1 ALL=(ALL) NOPASSWD: SUDO_ACCESS_LEVEL1"  >> /etc/sudoers
    echo "%sudo-group-level2 ALL=(ALL) NOPASSWD: SUDO_ACCESS_LEVEL2"  >> /etc/sudoers
    echo "%sudo-group-full-access ALL=(ALL) NOPASSWD: ALL"  >> /etc/sudoers
    fi
}

#check os 
if [[ $(cat /etc/os-release | egrep "centos|redhat|fedora|rhel|oracle|rocky") != "" ]]
then
yum install sssd realmd oddjob oddjob-mkhomedir adcli samba-common samba-common-tools krb5-workstation openldap-clients  -y
realm join -vvv --user=administrator ad.example.com

#call function
update_sssd_config

restrict_ssh_access_group

sudo_access_level_group

fi

Jenkins AD integration and access management

  • First take the back up of /var/jenkins_home/config.xml or take a snapshot if it’s vm.

Method 1: (all user have same admin access)

  • Manage jenkins > Configure Global Security > Active Directory
Domain name : example.com
Domain Controller : ad.exmaple.com
Bind DN : jenkins-svc-account@example.com
Bind Password : <jenkins-svc-account password>

Note: You may need to apply the setting and then click on Test Domain.

Else it will give Error: simple bind failed: ad.example.com:389 error

Also Domain Name(example.com) should be resolved from jenkins host or entry should be in /etc/hosts 

192.168.122.6 example.com

Method 2: Matrix-based security

Method3: Role based strategy(pending)

AD user and jenkins screenshot for reference

  • Check ldap port in powershell
tnc example.com -port 389

Test-NetConnection -ComputerName example.com -Port 389

Build haskell static binary with docker

Why?
– Reduce surface attack
– Reduce docker image size

hola.sh

{-# LANGUAGE OverloadedStrings #-}
import Web.Scotty

import Data.Monoid (mconcat)

main = scotty 3000 $
    get "/:word" $ do
        beam <- param "word"
        html $ mconcat ["<h1>Scotty, ", beam, " me up!</h1>"]

alpine.Dockerfile

FROM haskell:8 AS build
WORKDIR /opt
RUN cabal update
RUN cabal install --lib scotty
COPY hola.hs .
#RUN ghc --make -threaded hola.hs  -o hola
RUN ghc --make -threaded -optl-static -optl-pthread hola.hs -o hola

FROM alpine:3.15.0
RUN addgroup -S group1 && adduser -S user1 -G group1
USER user1
WORKDIR /opt
COPY --from=build /opt/hola .
EXPOSE 3000
CMD ["/opt/hola"]

More on haskell static binary –

DevOps questions

Docker

  • Explain Docker life cycle?
  • Explain/Write simple Dockerfile start to end.
  • What are layers in docker? how to reduce layer?
  • What is CMD and entry point? can we use both? When to use?
  • What is multistage dockerfile?
  • What’s Difference between ADD vs COPY, RUN vs CMD?
  • How to optimize docker image size?
  • How to optimize security in dockerfile?
  • Tools used for scanning docker file and image?
  • What is distroless image? when to use?
  • How to verify distroless image with cosign?
  • have you used tini ? why?
  • Can you run container with different kernel version than base kernel host?
  • What are the best practices to write Dockerfile?
  • what is docker buildkit?
  • What is docker-compose?
  • How container talks to other container in docker-compose?
  • Types of networks in docker-compose?
  • how to auto restart docker container?

Jenkins

  • Explain Your jenkins CICD pipeline?
  • Explain Jenkins DevSecOps Pipleine?
  • How you integrated sonarqube with your pipeline?
  • What are the ways we can start the build? OR type of build triggers?
  • How you add jenkins worker node? what are other ways to do?
  • How you manage credentials in Jenkins?
  • Diff between webhook and pool scm?
  • What plugins you used in Jenkins?
  • How do you take backup of Jenkins?
  • Have you written declarative or scripted pipeline? what’s difference?
  • How you pass env variable from one stage to another stage?

Kubernetes

  • What are the components of kuberntes?
  • What is pod and deployment, Daemonset
  • When do you use Daemonset?
  • What is sidecar container? when to use?
  • What is static pod?
  • How you have integrated CICD with Jenkins?
  • What is Kubeproxy? CoreDNS?
  • How you expose your service externally?
  • What will happen if k8 control node goes down?
  • have you upgraded your k8 cluster? what are the steps? explain
  • How you add security to k8 cluster?
  • How you scan your k8 deployment file?
  • Best practice to write k8 deployment file, explain componets?
  • What is taints, tolerance, node affinity, pod affinity , anti-affinity?
  • What is CNI? What CNI you have used? Why we choose different CNI?
  • Default scaling method available in kubernetes? What task are require to setup HPA?
  • What VPA? what kind of metrics we can use? from where?
  • How pod communicate with each other?
  • How kubernetes kube metrics works?
  • How you troubleshoot if pod is in pending/imagepullbackoff state?
  • what is service mesh ? when to use?
  • What is mTLS? how do you secure your pod to pod communication?
  • What is PV, PVC? what is storageclass?
  • If PV is 50GB and created 20GB PVC using that PV. can 30GB used for new PVC?
  • what is PVC accessModes and Reclaim Policy?

Terraform

  • what is terraform backend?
  • Where do you store your terraform state file? why?
  • Have you created custom module?
  • Explain your Iaac terraform pipeline

Ansible

  • What is ansible playbook?
  • What is ansible role?
  • What module you used in ansible?
  • What is register in ansible?
  • Difference between static and dynamic inventory?
  • How ansible works? how it do authentication?
  • What is ansible galaxy?
  • what is ansible vault? how you store your creds?
  • write playbook to install httpd and enabled it?
  • What is ansible tower?
  • what are handlers in ansible?

General questions

  • What do you mean by CI and CD?
  • What is continue development , continue deployment?
  • what is your git branch strategy? how do you resolve merge conflict?
  • How you optimize cost on aws? (amd, arm, cloudcustodian, aws lambda, s3 static website, s3 lifecycle)
  • How you will host static website?
  • Application is receiving millions of request ? how you will manage it?
  • What is blue-green deployment? tools required? strategy?
  • What is unit testing? how do you configure quality gates, code coverage in sonarqube?

AWS questions

  • AWS lambda time limit?
  • How many VPC and subnet we can create?
  • How do you secure your website, endpoints?
  • What is cloudfront distribution point?
  • How you can recover from wrong fstab configured ec2 which is not starting at all?

Linux questions

  • How do extend space in Linux? LVM? LVM striping?
  • RAID types?
  • Top 10 command that use for daily task?
  • SUID, GUID, Sticky Bit, special permission in linux?
  • which is faster cp or mv? why?
  • command not found? troubleshooting?
  • what is swap memory? why? how?
  • update time in linux?
  • linux boot process?

Add ubuntu WSL 2 in Visual code

  • Set default wls to ubuntu
wsl -l
wsl --set-default Ubuntu
wsl --set-default-version 2
  • change from wsl1 to wsl2
wsl -l -v 
wsl --set-version Ubuntu 2
  • chmod is not working?

create /etc/wsl.conf

[automount]
options = "metadata"
https://stackoverflow.com/questions/46610256/chmod-wsl-bash-doesnt-work

Kong smtp email configuration with AWS SES

update /etc/kong/kong.conf

smtp_mock=off
smtp_host=email-smtp.eu-west-1.amazonaws.com
smtp_port=465
smtp_username=${KONG_SMTP_USER}
smtp_password=${KONG_SMTP_PASSWORD}
smtp_ssl=on
smtp_domain=example.com
smtp_admin_emails=team1@example.com
admin_emails_from =Team1 <team1@example.com>
portal_invite_email=Team1 <team1@example.com>
portal_access_request_email=Team1 <team1@example.com>
portal_approved_email=on
portal_emails_from=Team1 <team1@example.com>
portal_emails_reply_to=Team1 <team1@example.com>

Note : team1@example.com should verified in AWS and It’s should be below format

Team1 <team1@example.com>