Skip to content

PR Environments

Resetting database

With a local kubectl client authenticated to the indevets-sandbox cluster, you can use the script/reset-pr-env command included in the repository to completely reset any PR environment’s database.

This can also be useful when a PR environment has been accidentally reset between deploys and lost its database.

It accepts the PR number as an argument, for example:

./script/reset-pr-env 271

Troubleshooting

  • Check GitHub’s Actions section: https://github.com/IndeVets/core/actions
    • Find the k8s deploy workflow for your PR #
    • Open the log output
    • Did the job fail?
      • Expand the step that failed and review the command output
    • Do you need to change your code?
      • Leave this failed job behind and push some new commits to trigger a new job
    • Do you need to change something besides your code? (e.g. fix QuickBooks tokens)
      • Re-run the failed job with the button inside GitHub
    • When in doubt: the only risk of re-running the last failed job on a PR is the time you might spend waiting to see it fail again

  • Was the last deploy successful, but now it’s down?

    • Is https://staging.k8s-sandbox.indevets.com and other recent PR environments down also?

      • Get Nafis or Chris or Linode to check out the cluster

      • Try running the magic fixer for kubernetes#80968:

        export KUBECONFIG=~/Downloads/indevets-sandbox-kubeconfig.yaml
sudo hab pkg install jarvus/kube-pod-update-status
hab pkg exec jarvus/kube-pod-update-status fix-all-pods

    • Is just this environment or just this and some other odd ones down?

      • Force a new deployment without changing any code:

        git commit --amend --no-edit
git push -f

  • Are deployments failing while setting up the database because of QuickBooks authentication?