JQ

Example-wise the jq manpage is not really helpful. Let’s document some simple examples here…

To test queries live use https://jqplay.org/

Output Formatting

If you do only care about output formatting (pretty print) run

jq . my.json

Note: for redirection you need to pass a filter too to avoid a syntax error:

jq . my.json > output.json

Simple Extraction

Consider this example document

{
    "timestamp": 1234567890,
    "report": "Age Report",
    "results": [
        { "name": "John", "age": 43, "city": "TownA" },
        { "name": "Joe",  "age": 10, "city": "TownB" }
    ]
}

To extract top level attributes “timestamp” and “report”

jq '.[] | {timestamp,report}'

To extract name and age of each “results” item

jq '.results[] | {name, age}'

Filter this by attribute

jq '.results[] | select(.name == "John") | {age}'      # Get age for 'John'
jq '.results[] | select(.name | contains("Jo"))'       # Get complete records for all names with 'Jo'
jq '.results[] | select(.name | test("Joe\s+Smith"))'  # Get complete records for all names matching PCRE regex 'Joe\+Smith'

Changing values

Merging/overwriting keys

echo '{ "a": 1, "b": 2 }' |\
jq '. |= . + {
  "c": 3
}'

Adding elements to lists

echo '{ "names": ["Marie", "Sophie"] }' |\
jq '.names |= .+ [
   "Natalie"
]'   

Handle Empty Arrays

When you want to iterate and an array you access is empty you get something like

jq: error (at <stdin>:3): Cannot iterate over null (null)

To workaround the optional array protect the access with

select(.my_array | length > 0)

Using jq in Shell Scripts

From https://www.terraform.io/docs/providers/external/data_source.html

Parsing JSON into env vars

To fill environment variables from JSON object keys (e.g. $FOO from jq query “.foo”)

export $(jq -r '@sh "FOO=\(.foo) BAZ=\(.baz)"')

To make a bash array read -a bash_array < <(jq -r .|arrays|select(.!=null)|@tsv)

JSON template using env vars

To create proper JSON from a shell script and properly escape variables:

jq -n --arg foobaz "$FOOBAZ" '{"foobaz":$foobaz}'

URL Encode

Quick easy way to url encode something

date | jq -sRr @uri