Sunday, December 9, 2012

Creating a Go Library: Part 1


This tutorial will follow my progress working on a library that would handle working with dynamic JSON data. Go is a great language for decoding JSON when the data layout is fixed, but since Go is a static language, dealing with JSON that is dynamic can be less intuitive than when working with dynamic languages like PHP, Ruby, or Python.  This tutorial assumes that you have some working knowledge of Go.  If you have no prior knowledge of Go, go here and here.

I think Go is lacking in regards to the amount of intermediate and advanced examples that show how Go works solving actual problems instead of a "10000 foot view"of Go.  I have great respect for the Go team and in particular, I think Andrew Gerrand is doing a great job of representing Go accross the globe, but if I see one more video of Andrew demoing chat roulette, I will commit Seppuku

Taking a whack at it

This is the sample JSON data from the Github API I am working with.  And here is a sample JSON record.

Looking at the events data, I found that the payload object is different depending on the "type" of event.  Immediately, I wanted to go back to my PHP roots and use $event = json_decode($source) and this problem would be solved, but the following example uses the most basic Go code.

The problem with this is you can only access top level data items.  For instance, I can access the payload by using m["type"] but trying to get m["payload"]["description"] will not compile because an interface does not have any indexes.  So what can we do to clean this up?  I created a couple of structs and some basic functions to help me decode the JSON.  I wanted a JSONCollection struct which contained the entire document and JSONObject which would be the struct used to read the results.

This still has the same limitation as before.  I can use "Get" to find a top level attribute, but If I want a nested attribute, I have to type cast the interface to a map again and then grab the description out of that map.  I need to come up with a better solution. The goal is to have syntax like the following.

This will give us a cleaner syntax to get at nested variables. So let's fix the code to allow this.

This gives a start to our library.  Now we can retrieve strings of data (and nested data) with a minimal amount of Go code.  Here is a an example as to how we would use this library.

Current Limitations

The library uses only strings, so if you have an integer or float, those values will have trouble converting to strings (I will fix this in a later blog post).  You will have trouble traversing nested slices as that this will cause an error.  Currently the example is read-only, as that was all that was initially needed.

Coming Up on Personal Ramblings

I want to make this a feature complete library, so here is a list of things that I want to add to this library.  
  • Get JSON Object by Position
  • Get JSON Objects by using limit/offset
  • Filter JSON Results
  • Handling different variable types
  • Creating Tests
  • Creating a better Iterator
  • Using Godoc
  • Using GoFmt
If you feel anything is missing, feel free to comment below.  When completed I will be releasing the source for this library on Github.


  1. Hello. This post couldn’t be written any better! Reading this post reminds me of my previous roommate. He always kept chatting about this. I will forward this page to him. Fairly certain he will have a good read. Thank you for sharing.
    AWS Training in Bangalore |Best AWS Training Institute in Bangalore BTM, Marathahalli
    AWS Training in Chennai | AWS Training Institute in Chennai Velachery, Tambaram, OMR


  2. When I initially commented, I clicked the “Notify me when new comments are added” checkbox and now each time a comment is added I get several emails with the same comment. Is there any way you can remove people from that service? Thanks.
    Amazon Web Services Training in OMR , Chennai | Best AWS Training in OMR,Chennai
    Amazon Web Services Training in Tambaram, Chennai|Best AWS Training in Tambaram, Chennai

  3. Excellant post!!!. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it.
    online Python certification course | python training in OMR | Python training course in Chennai

  4. Great Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us and I never get bored while reading your article because, they are becomes a more and more interesting from the starting lines until the end.
    Devops Training courses
    Devops Training in Bangalore
    Best Devops Training in pune
    Devops interview questions and answers

  5. It is better to engaged ourselves in activities we like. I liked the post. Thanks for sharing.
    Java training in Chennai | Java training in Bangalore

    Java online training | Java training in Pune

  6. Thank you for this post. Thats all I are able to say. You most absolutely have built this blog website into something speciel. You clearly know what you are working on, youve insured so many corners.thanks
    Data Science course in rajaji nagar | Data Science with Python course in chenni

    Data Science course in electronic city | Data Science course in USA

    Data science course in pune | Data science course in kalyan nagar

  7. This is most informative and also this post most user friendly and super navigation to all posts... Thank you so much for giving this information to me.. 

    best rpa training in chennai | rpa online training |
    rpa training in chennai |
    rpa training in bangalore
    rpa training in pune
    rpa training in marathahalli
    rpa training in btm

  8. Nice blog..! I really loved reading through this article. Thanks for sharing such an amazing post with us and keep blogging...Well written article Thank You for Sharing with Us pmp training institute in chennai | pmp training class in chennai | pmp training near me | pmp training courses online | pmp training fee