Enabling debug information on terraform using TF_LOG

2 min read | by Jordi Prats

Sometimes with terraform you might end up with some meaningless error that does not provide any clue what's going on. So instead or just trying to guess that have changed; we can enable some traces in terraform to try to make sense what's going on

Error: AccessDenied: Access Denied
  status code: 403, request id: 8FCD12996B11F5F8, host id: xMF1Gs+VIEpxpk+1Og6UtchyT10K+mRWFe2IUZ8gqG13KbsRm0L8nRw8udzkqEVJagg8+RMpY3M=

To tell terraform to give us some more info of what's going on we'll have to use the TF_LOG environment variable. We can set it to any of the log levels: TRACE, DEBUG, INFO, WARN or ERROR

If it is set to something other that, it will default to TRACE which is also the most verbose. So if we run terraform like this:

TF_LOG="trace" terraform plan

We are going to get a lot of traces to dig into:

---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: dynamodb.us-west-2.amazonaws.com
User-Agent: aws-sdk-go/1.30.12 (go1.12.13; linux; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.29
Content-Length: 184
Accept-Encoding: identity
Authorization: AWS4-HMAC-SHA256 Credential=666A3LJTGU66FBFOPQJS/20210210/us-west-2/dynamodb/aws4_request, SignedHeaders=accept-encoding;content-length;content-type;host;x-amz-date;x-amz-security-token;x-amz-target, Signature=9da715cf4cd80be5f211909428faa18fb09d8a76ac05490845c018cd7224a6a6
Content-Type: application/x-amz-json-1.0
X-Amz-Date: 20210210T100656Z
X-Amz-Security-Token: FwoGZXIvYXdzELv//////////wEaDADkXLaiHsjRK74RZiKjArUMW1EsjjpnrztmjkOd/hD0tujLaErRG5HJU3KwShs8T4OKkSldQb2HaebMWDVNoId5jH6+RHJDW2BvGumPHd0L8tv0Ut3O3AvVa5F63opczeq084rjHkvawyzrRKUHOlVKYzre3zMK0PbwuSP/wS/g8oflNSrz7y29h9HDKKwgTbYg5BG8n5m1pMnHQ7vMpz9vlwW8WQxTcswMrZuEwhLwnqhYAf9iOpyCMqEU/
fx161ergvT3Z+odkxKIXl9ObTB40Bssgn149zUNjd2eY2SGUHHjVeCJQ2TroKkvRgTUl1pHLKfahchZRlVT3WzPqOi/JAmlzu0+sKbl2SopYArLnOBfr4WNqQBDlj10hmmaQ/dZVa+hjosm482DwumvesL/+ijI2o6BBjIqNLliZbQ2eaWde60KdKptyxuVjOX/YXJtYW5kbyBzdWJub3JtYWwKItVg

X-Amz-Target: DynamoDB_20120810.GetItem

{"ConsistentRead":true,"Key":{"LockID":{"S":"demo/test/sample.tfstate"}},"ProjectionExpression":"LockID, Info","TableName":"sample_statelock"}
-----------------------------------------------------

Error: AccessDenied: Access Denied
  status code: 403, request id: 8FCD12996B11F5F8, host id: xMF1Gs+VIEpxpk+1Og6UtchyT10K+mRWFe2IUZ8gqG13KbsRm0L8nRw8udzkqEVJagg8+RMpY3M=

For each situation and context we will have to tackle it's output differently, just as you would do with strace and a rogue Linux process, but a good start could be taking a look at the API responses terraform is getting like follows:

TF_LOG="trace" terraform plan >/tmp/tferror 2>&1 | grep RESPONSE -A1
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
--
(...)
---[ RESPONSE ]--------------------------------------
HTTP/1.1 403 Forbidden
--
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
--
---[ RESPONSE ]--------------------------------------
HTTP/1.1 403 Forbidden
--
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
--
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
--
(...)

Posted on 09/03/2021

Categories