Building JSON on the Command Line is Obnoxious

As a professional “gluer of computer programs”, I’ve found myself needing to interact with JSON quite a bit. I also often find myself needing to interact with JSON on the command line. Unsurprisingly jq is an integral part of my toolkit, as I’m sure it is for many of my ilk.

While jq is a masterclass on how to deconstruct or re-organize JSON on the command line, a frequent pain point for me is building JSON via the command line.

Your first option is to just straight echo some text:

echo '{"some_key":{"sub_key":"its value"},"another_key":"another value"}' | jq '.'

this produces, without error:

{
  "some_key": {
    "sub_key": "its value"
  },
  "another_key": "another value"
}

Nice. Where this quickly breaks down is when you start needing to do fancier things with quotes, or variable substitution. You can get away with this sometimes, but inevitably you end up needing to escape every… single… " in your entire echo command. Obnoxious. Yes, there are some workarounds, but it gets messy fast.

Next, you have the heredoc in BASH, so something like:

SUB_ME="its value"
jq '.' << EOF
{
  "some_key": {
    "sub_key": "$SUB_ME"
  },
  "another_key": "another value"
}
EOF

This actually works pretty well, but this can be really tricky to deal with too, especially depending on how you need to pass this JSON on. This will or will not be super annoying depending on what needs to ingest next. It’s also a bit cumbersome to type out JSON this way, but at least you can do newlines easily enough and do indenting the way you want to make it nicer. It’s still not a nice experience. I much prefer to avoid it if at all possible.

My Solution

I’ve been wondering how to deal with this in day to day work, and have noticed different ways of handling it. Where this is most annoying is when dealing with command line tools that accept a string of JSON as an argument. The alternative I’ve seen in CLI tool design is to instead take as an argument the filename to a .json. This can be as annoying, but for other reasons.

If only there were a way to easily type out JSON on the command line. In my search for such a tool I have come across a couple solutions, ranging from some nastiness using arguments passed into jq, to a super purpose-built tool called jo.

Looking through these solutions, the ergonomics of a different tool kept coming to the front of my mind.

“Why can’t I build JSON on the command line like I can build JSON bodies in httpie?”

If you’ve never used httpie, it’s a more modern curl with better ergonomics. It’s not quite as powerful, but it has a lovely syntax for providing headers, http parameters, and json body content on the command line. It can even do nested json!

It’s perfect, it’s ergonomic, what’s not to love?! Where is my jo-like single-purpose CLI tool to build JSON with this clean syntax?

To my chagrin, I’ve not been able to find one. I’ve wondered, should I think about building my own? I actually don’t use httpie itself, but xh, a rust clone. Maybe I can figure out which library they’re using in xh to parse this syntax, that would be awesome.

Then, I discovered this functionality, hinted at in httpie’s docs above:

xh --offline --print=B fake.url "some_key[sub_key]=its value" "another_key=another value"

The key here is the --offline and --print=B, which tells xh to not send anything to fake.url and to just print the body that it would have sent. 🎉

This guy is deserving of an alias for sure:

alias xhjb="xh --offline --print=B fake.url" # xh json builder