• Using ephemeral containers

    3 min read

    kubernetes kubectl debug troubleshooting ephemeral containers

    Starting Kubernetes 1.23, ephemeral containers are enabled by default (in beta though). Using ephemeral containers we can now troubleshoot pods by deploying a temporary container into it with extra privileges or binaries to use

    22/08/2022

    Read more...
  • Troubleshooting a Pod by changing it's command

    3 min read

    kubernetes Pod troubleshooting

    When a container in a Pod is crashing sometimes it's logs are not enough to fully understand what's going on. One way to approach this situation the command it runs to something that won't make Kubernetes restart the container: For example a sleep command

    08/03/2022

    Read more...
  • How to debug a crossplane provider

    4 min read

    crossplane kubernetes troubleshooting debug

    To be able to setup a crossplane provider there are some pieces that need to be aligned to be able to use it. For example, if we want to setup the AWS provider using an IAM Role for ServiceAccount. If something is missaligned, we might end up with an error while creating resources that doesn't really clarify what's the actual error:

    $ kubectl describe bucket.s3.aws.crossplane.io/test-bucket
    Name:         test-bucket
    Namespace:    
    Labels:       <none>
    Annotations:  crossplane.io/external-name: pet2cattle-demo
    API Version:  s3.aws.crossplane.io/v1beta1
    Kind:         Bucket
    Metadata:
    (...)
    Spec:
    (...)
      Provider Config Ref:
        Name:  aws-provider
    Status:
      At Provider:
        Arn:  
      Conditions:
        Last Transition Time:  2022-02-22T21:43:23Z
        Message:               observe failed: failed to query Bucket: api error MovedPermanently: Moved Permanently
        Reason:                ReconcileError
        Status:                False
        Type:                  Synced
    Events:
      Type     Reason                         Age               From                                 Message
      ----     ------                         ----              ----                                 -------
      Warning  CannotObserveExternalResource  7s (x6 over 36s)  managed/bucket.s3.aws.crossplane.io  failed to query Bucket: api error MovedPermanently: Moved Permanently
    

    23/02/2022

    Read more...
  • Packer templates: Add more verbosity to shell provisioners

    2 min read

    If we use shell scripts as provisioners with packer errors can be confusing:

    ==> amazon-ebs: Terminating the source AWS instance...
    ==> amazon-ebs: Cleaning up any extra volumes...
    ==> amazon-ebs: No volumes to clean up, skipping
    ==> amazon-ebs: Deleting temporary security group...
    ==> amazon-ebs: Deleting temporary keypair...
    Build 'amazon-ebs' errored: Error removing temporary script at /tmp/script_9722.sh!
    
    ==> Some builds didn't complete successfully and had errors:
    --> amazon-ebs: Error removing temporary script at /tmp/script_9722.sh!
    
    ==> Builds finished but no artifacts were created.
    

    We can tell there is an error, but it's hard to tell what's going on

    26/01/2022

    Read more...
  • Troubleshooting on kubernetes using netshoot

    1 min read

    pod troubleshooting netshoot kubernetes nicolaka/netshoot

    One of most common issues we might have while troubleshooting an issue on a kubernetes cluster is to actually not having the right tool for the job. Containers usually have the bare minimum set of tools (actually, this is how it's supposed to be) and libraries required to do it's job: So they are not designed for troubleshooting

    02/02/2021

    Read more...

From pet to cattle
Treat your kubernetes clusters like cattle, not pets