Delegate Amazon EKS operations to your DevOps team

Kubernetes is an elegant tool for running and scaling your container applications. The other side of the coin is that it has got a steep learning curve to bootstrap from scratch and maintain the operability across its various components. But the good news is that you can save some effort and time by leveraging managed Kubernetes services from some public cloud providers such as AWS, Google, Microsoft. And your K8s cluster is ready to go with few clicks on their administration GUI.

In particular, Amazon Elastic Kubernetes Service (Amazon EKS) eases the management of the availability and scalability of Kubernetes control plane nodes. These nodes act like the brain of Kubernetes and are responsible for scheduling containers, managing the availability of applications, storing cluster data, and other key tasks. By default, only the AWS IAM entity who created the EKS cluster will be granted the cluster administrator rights (added to “system:masters” group). In other words, AWS users other than cluster creator won’t be able to perform any K8s operations against the cluster on the command line. Though the hard core administration tasks such as software upgrade, underlying infrastructure provisioning is usually handled by the cluster creator, would it be good if some tasks can be delegated to other DevOps members ? For instance, you may want your different development teams to manage their own Kubernetes namespaces without going to the cluster admin every time for namespace resource allocation.

In EKS, the following is used to bridge AWS user identity to K8s user identity for managing namespaces permission.

In this tutorial, I am going to setup AWS resources and permission to enable a DevOps team to operate a Kubernetes namespace.

Assume the DevOps team is already equipped with an AWS login. The following commands are based on a UNIX environment.

1. We need to create an IAM role that is to be assumed by the DevOps team for delegation. Details can be found at https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user.html

2. Have the DevOps team member assume the IAM role in CLI. The following command will let you assume an IAM role and inject the credentials into AWS environment variables.

eval $(aws sts assume-role --role-arn arn:aws:iam::yourAccountID:role/yourIAMRoleName --role-session-name test | jq -r '.Credentials | "export AWS_ACCESS_KEY_ID=\(.AccessKeyId)\nexport AWS_SECRET_ACCESS_KEY=\(.SecretAccessKey)\nexport AWS_SESSION_TOKEN=\(.SessionToken)\n"')

3. Configure the IAM user’s kubectl to use this IAM role when talking to Kubernetes API by updating the kubeconfig file.

$ aws eks update-kubeconfig --name yourClusterName --role-arn arn:aws:iam::yourAccountID:role/yourIAMRoleName

4. EKS comes with a default clusterrole “admin”. Clusterrole is cluster wide and applies to all resources. Role is namespace specific. As we are trying to delegate a namespace only to the DevOps team, we create a new role-based access control (RBAC) role by K8s yaml file (e.g. role.yaml) for their use in “example_namespace”

For more information on RBAC, please visit https://kubernetes.io/docs/reference/access-authn-authz/rbac/

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: k8s-test-role
  namespace: example_namespace
rules:
  - apiGroups:
      - ""
      - "apps"
      - "batch"
      - "extensions"
    resources:
      - "configmaps"
      - "cronjobs"
      - "deployments"
      - "events"
      - "ingresses"
      - "jobs"
      - "pods"
      - "pods/attach"
      - "pods/exec"
      - "pods/log"
      - "pods/portforward"
      - "secrets"
      - "services"
    verbs:
      - "create"
      - "delete"
      - "describe"
      - "get"
      - "list"
      - "patch"
      - "update"
      - "watch"

Modify the resources list and verbs according to your needs. A list of verbs can be found at https://kubernetes.io/docs/reference/access-authn-authz/authorization/#review-your-request-attributes

5. Apply the yaml file to create the RBAC role.

$ kubectl apply -f role.yaml

6. Next create a role binding for this new RBAC role by applying the following rolebinding.yaml

There is no need to create the k8s-test-user separately. This user is for K8s internal use only.

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: k8s-test-rolebinding
  namespace: example_namespace
subjects:
- kind: User
  name: k8s-test-user
roleRef:
  kind: Role
  name: k8s-test-role
  apiGroup: rbac.authorization.k8s.io$ kubectl apply -f rolebinding.yaml

7. Finally we do the mapping by associating the IAM role with the k8s-test-user.

$ eksctl create iamidentitymapping --cluster yourClusterName --arn arn:aws:iam::yourAccountID:role/yourIAMRoleName --username k8s-test-user

8. The new IAM role is granted permission to operate the “example_namespace”. You can have your DevOps team try the kubectl command against the namepsace. They should be able to operate and manage their namespace.

In my next article, I will talk about how a K8s resource assumes an IAM role to perform AWS operations.

Be sure to visit us at https://airwalkconsulting.com/ if you want to know more about digital transformation.