Changing the last character of a file

I want to continuously write json objects to a file. To be able to read it, I need to wrap them into an array. I don't want to read the whole file, for simple appending. So what I' doing now:

I want to continuously write json objects to a file. To be able to read it, I need to wrap them into an array. I don't want to read the whole file, for simple appending. So what I' doing now:

comma := []byte(", ")
    file, err := os.OpenFile(erp.TransactionsPath, os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0666)
    if err != nil {
        return err
    }
    transaction, err := json.Marshal(t)
    if err != nil {
        return err
    }
    transaction = append(transaction, comma...)
    file.Write(transaction)

But with this implementation I will need to add []scopes by hand(or via some script) before reading. How can I add an object before closing scope on each writing?

How to work with JSON data in Go?

How to work with JSON data in Go?

This article covers the encoding/json package which handles conversion between JSON and Go types (as per RFC 7159).

This article covers the encoding/json package which handles conversion between JSON and Go types (as per RFC 7159). Let's jump right in!

the code is available in this GitHub repo

Converting a Go type into JSON

Marshal

A common option is to use the Marshal function whose signature is:

    func Marshal(v interface{}) ([]byte, error)

Here is an example:

func main() {
    profile := Profile{Email: "[email protected]", Username: "abhirockzz", Blogs: []Blog{
        Blog{BlogName: "devto", URL: "https://dev.to/abhirockzz/"},
        Blog{BlogName: "medium", URL: "https://medium.com/@abhishek1987/"},
    }}
    jsonData, err := json.Marshal(&myprofile)
    //jsonData, err := json.MarshalIndent(&profile, "", " ")
    if err != nil {
        panic(err)
    }
    fmt.Println(string(jsonData))
}

you can use MarshalIndent (commented) to indent the JSON output

Encoder

While Marshal deals in byte arrays ([]byte), an Encoder is generic and lets you work with an io.Writer which you can define to be the sink for JSON data i.e. you can specify any type which implements the io.Writer interface e.g. this could be standard out (os.Stdout), or an HTTP response (http.ResponseWriter), etc.

func main() {
    profile := Profile{Email: "[email protected]", Username: "abhirockzz", Blogs: []Blog{
        Blog{BlogName: "devto", URL: "https://dev.to/abhirockzz/"},
        Blog{BlogName: "medium", URL: "https://medium.com/@abhishek1987/"},
    }}

    encoder := json.NewEncoder(os.Stdout)
    err := encoder.Encode(&profile)
    if err != nil {
        panic(err)
    }
}

Use NewEncoder to specify the io.Writer. When you call Encode, the conversion takes place and the JSON is writer to io.Writer you specified.

Here is an example of how it would work with an HTTP response:

func main() {
    profile := Profile{Email: "[email protected]", Username: "abhirockzz", Blogs: []Blog{
        Blog{BlogName: "devto", URL: "https://dev.to/abhirockzz/"},
        Blog{BlogName: "medium", URL: "https://medium.com/@abhishek1987/"},
    }}

    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        encoder := json.NewEncoder(w)
        err := encoder.Encode(&profile)
        if err != nil {
            panic(err)
        }
    })

    http.ListenAndServe(":8080", nil)
}

Converting JSON data to a Go type

Unmarshal

Given JSON in form of a byte array, Unmarshal stores the JSON parsing result to a pointer of the Go data type you specify (typically a struct)

func Unmarshal(data []byte, v interface{}) error

Here is a simple example:

func main() {
    jsonData := `{"email":"[email protected]","username":"abhirockzz","blogs":[{"name":"devto","url":"https://dev.to/abhirockzz/"},{"name":"medium","url":"https://medium.com/@abhishek1987/"}]}`

    var profile Profile
    err := json.Unmarshal([]byte(jsonData), &profile)
    if err != nil {
        panic(err)
    }
    ...
}

Decoder

Decoder provides a generic way of unmarshalling JSON data by allowing you to specify the source of JSON input in the form of an io.Reader as opposed to a byte array.

func main() {
    jsonData := `{"email":"[email protected]","username":"abhirockzz","blogs":[{"name":"devto","url":"https://dev.to/abhirockzz/"},{"name":"medium","url":"https://medium.com/@abhishek1987/"}]}`

    jsonDataReader := strings.NewReader(jsonData)
    decoder := json.NewDecoder(jsonDataReader)

    var profile Profile
    err := decoder.Decode(&profile)
    if err != nil {
        panic(err)
    }
    ...
}

We start by creating an io.Reader from the JSON string data using a shortcut strings.NewReader. We can then instantiate the decoder using NewDecoder and simply use decode with a pointer to the Profile struct to which the result is stored

JSON Tutorial For Beginners | What is JSON | Learning JSON with JavaScript

JSON Tutorial For Beginners | What is JSON | Learning JSON with JavaScript

JSON Tutorial For Beginners | What is JSON | Learning JSON with JavaScript

Explore JSON and how JavaScript Objects can be used to access data within JSON data format and output to your web page

Guide to learning how to use JavaScript Objects and JSON data. JSON is the most popular format for data exchange between applications. If you are interested in connected to a web API chances are its JSON formatted. Learn to use AJAX to connect and bring JSON data into your JavaScript!

This course shows you how to work with JSON formatted data, output content, loop JSON data, Parse JSON and a whole lot more.

JSON (JavaScript Object Notation) is a syntax for data. JSON is easier to use than XML and human readable. Most modern web APIs output data in JSON formats. It's a lightweight data interchange format that is quickly becoming the default format for data exchange on internet today! JSON is lightweight, language independent and easy to read and write. JSON is better than XML and more popular!

Within the lessons of this course we will explore

  • JavaScript data types used to hold variables and how they work
  • JSON and how to write JSON data
  • How to add values into a JSON object
  • Accessing JSON data and bringing it into JavaScript
  • JavaScript JSON parse and stringify methods
  • Adding JSON to local storage
  • Retrieving back data within JSON formats, updating and working with JSON
  • Connecting to a web API using fetch
  • Retrieving JSON data from a web API and outputting the results into your web page
  • Iterating threw multiple results from an API
  • Google Spreadsheet data as JSON and how to practice retrieving data
  • All of the source code and resources are in included
  • Explore different methods of working with the JSON data stringify and parsing
  • How JavaScript objects can use Arrays to hold multiple items
  • How JavaScript arrays work and store data

Thanks for reading

If you liked this post, please do share/like it with all of your programming buddies!

Follow us on Facebook | Twitter

Further reading about JavaScript and JSON

The Complete JavaScript Course 2019: Build Real Projects!

JavaScript Programming Tutorial | Full JavaScript Course for Beginners 2019

The complete beginner’s guide to JSON

The Complete Guide to JSON Web Tokens

JWT Fundamentals for Beginners

Best JavaScript Frameworks, Libraries and Tools to Use in 2019

New ES2019 Features Every JavaScript Developer Should Know

How to work with JSON in Golang

How to work with JSON in Golang

In this article, I will show you the basics of how to parse and return JSON in Go while also providing you with an understanding of the process.

Introduction

In this article, I will show you the basics of how to parse and return JSON in Go while also providing you with an understanding of the process.

First, let’s look at how JSON is parsed in other languages specifically dynamically typed ones, I’m going to use python as an example.

##############
RETURNING JSON 
##############
import json
a Python object (dict):

p_dict = {
"name": "John",
"age": 30,
"city": "New York"
}

convert into JSON:

y = json.dumps(p_dict)

the result is a JSON string:

print(y)
##############
CONSUMING JSON
##############
import json

some JSON:

j_str = '{ "name":"John", "age":30, "city":"New York"}'

parse x:

y = json.loads(j_str)

the result is a Python dictionary:

print(y["age"])

The code looks relatively simple, in the case of returning JSON we have a python dictionary “p_dict” which we can convert to a JSON string using json.dumps(p_dict) . When consuming JSON, we have a JSON string “j_str” which we can convert to a python diction using json.loads(j_str) .

Now parsing a format like JSON in a statically typed language like Go presents a bit of a problem. Go is statically typed which means the type of every variable in the program needs to be known at compile time. This means that you as the programmer must specify what type each variable is. Other languages (python) offer some form of type inference, the capability of the type system to deduce the type of a variable. The main advantage of a statically typed language is that all kinds of checking can be done by the compiler, and therefore a lot of trivial bugs are caught at a very early stage. A simple code example may clarify.

PYTHON
x = 3
GOLANG
var x int = 3

When parsing the JSON anything could show up in the JSON body, so how does the compiler know how to set up memory given it doesn’t know the types?

There are two answers to this. One answer when you know what your data will look and one answer for when you don’t. We will cover the first option initially as this is the most common situation and leads to the best practices.

When the data types in the JSON is known you should parse the JSON into a struct you’ve defined. Any field which doesn’t fit in the struct will just be ignored. We’ll explore this option in both returning and consuming JSON.

Returning JSON

Let’s say we want to return the following JSON object.

{
"key1": "value 1",
"key2": "value 2"
}

The code below shows how this is done, let’s talk through it.

package main

import (
"encoding/json"
"fmt"
)

type JSONResponse struct {
Value1 string json:"key1"
Value2 string json:"key2"
}

func main() {

jsonResponse := JSONResponse{
	Value1: "Test value 1",
	Value2: "Test value 2",
}

fmt.Printf("The struct returned before marshalling\n\n")
fmt.Printf("%+v\n\n\n\n", jsonResponse)

// The MarshalIndent function only serves to pretty print, json.Marshal() is what would normally be used
byteArray, err := json.MarshalIndent(jsonResponse, "", "  ")

if err != nil {
	fmt.Println(err)
}

fmt.Printf("The JSON response returned when the struct is marshalled\n\n")
fmt.Println(string(byteArray))

}

  • Lines 3–6: We import the encoding/json and fmt packages for parsing the JSON and printing the results.
  • Lines 8–11: The JSONReponse struct is the Go variable representation of the JSON in the same way the python dictionary was the python variable representation of the JSON.
  • Lines 8–11: Note in JSONReponse we only declare types no values, we also add “Go Tags” json: "key1" which let's Go know what the associated key is in the JSON object.
  • Lines 15–18: jsonResponse is an instantiation of the JSONResponse struct and populates the values.
  • Lines 20–21: We print the jsonResponse variable to the console so we can contrast it with the JSON which is later printed.
  • Lines 23–24: We Marshal the variable jsonResponse which is of type JSONResponse this converts it into a byte array and takes into consideration the mappings declared in the “Go Tags”. The returned byte array is stored in the byteArray variable.
  • Lines 26–28: We run an error check on the Marshal function and print the error if it is not nil.
  • Lines 30–31: We cast the byteArray to a string and print it, we can now view the difference between the printed struct and the printed JSON.

Please run the code here to see the results, play with the code also it’s the best way to learn https://play.golang.org/p/gaBMvz21LiA.

Nested JSON

Now let’s look at a more complex JSON object with nested items

{
"key1": "value 1",
"key2": "value 2",
"nested": {
"nestkey1": "nest value 1",
"nestkey2": "nest value 2"
}
}

The code below shows how this is done, let’s discuss what’s changed.

package main

import (
"encoding/json"
"fmt"
)

type JSONResponse struct {
Value1 string json:"key1"
Value2 string json:"key2"
Nested Nested json:"nested"
}

type Nested struct {
NestValue1 string json:"nestkey1"
NestValue2 string json:"nestkey2"
}

func main() {

nested := Nested{
	NestValue1: "nest value 1",
	NestValue2: "nest value 2",
}

jsonResponse := JSONResponse{
	Value1: "value 1",
	Value2: "value 2",
	Nested: nested,
}

// Try uncommenting the section below and commenting out lines 21-30 the result will be the same meaning you can declare inline

// jsonResponse := JSONResponse{
// 	Value1: "value 1",
// 	Value2: "value 2",
// 	Nested: Nested{
// 		NestValue1: "nest value 1",
// 		NestValue2: "nest value 2",
// 	},
// }

fmt.Printf("The struct returned before marshalling\n\n")
fmt.Printf("%+v\n\n\n\n", jsonResponse)


// The MarshalIndent function only serves to pretty print, json.Marshal() is what would normally be used
byteArray, err := json.MarshalIndent(jsonResponse, "", "  ")

if err != nil {
	fmt.Println(err)
}

fmt.Printf("The JSON response returned when the struct is marshalled\n\n")
fmt.Println(string(byteArray))

}

  • Line 11: Rather than declaring Nested as type string we declared Nested as type Nested which is a type we are about to create.
  • Line 14–17: We declared the Nested type struct which we have used in JSONResponse following the format of the JSON we want to return.
  • Line 21–30: We instantiate both the nested variable of type Nested and the jsonResponse variable of type JSONResponse which in turn references the nested variable we just declared.

Those are the key differences between this example and the previous one. Again run the code here to test it and make some of the changes suggested in the comments https://play.golang.org/p/GcRceKe1jC-.

Arrays in JSON

Finally, let’s look at returning a JSON object that contains an array.

{
"key1": "value 1",
"key2": "value 2",
"nested": {
"nestkey1": "nest value 1",
"nestkey2": "nest value 2"
},
"arrayitems": [
{
"iteminfokey1": "item info 1 array index 0",
"iteminfokey2": "item info 2 array index 0"
},
{
"iteminfokey1": "item info 1 array index 1",
"iteminfokey2": "item info 2 array index 1"
}
]
}

The code below shows how this is done, again let’s discuss what has changed.

package main

import (
"encoding/json"
"fmt"
)

type JSONResponse struct {
Value1 string json:"key1"
Value2 string json:"key2"
Nested Nested json:"nested"
ArrayItems []ArrayItem json:"arrayitems"
}

type Nested struct {
NestValue1 string json:"nestkey1"
NestValue2 string json:"nestkey2"
}

type ArrayItem struct {
ItemInfo1 string json:"iteminfokey1"
ItemInfo2 string json:"iteminfokey2"
}

func main() {

arrayItems := []ArrayItem{
	ArrayItem{
		ItemInfo1: "item info 1 array index 0",
		ItemInfo2: "item info 2 array index 0",
	},
	ArrayItem{
		ItemInfo1: "item info 1 array index 1",
		ItemInfo2: "item info 2 array index 1",
	},
}

nested := Nested{
	NestValue1: "nest value 1",
	NestValue2: "nest value 2",
}

jsonResponse := JSONResponse{
	Value1:     "value 1",
	Value2:     "value 2",
	Nested:     nested,
	ArrayItems: arrayItems,
}

fmt.Printf("The struct returned before marshalling\n\n")
fmt.Printf("%+v\n\n\n\n", jsonResponse)

// The MarshalIndent function only serves to pretty print, json.Marshal() is what would normally be used
byteArray, err := json.MarshalIndent(jsonResponse, "", "  ")

if err != nil {
	fmt.Println(err)
}

fmt.Printf("The JSON response returned when the struct is marshalled\n\n")
fmt.Println(string(byteArray))

}

  • Line 12: We add ArrayItems to our JSONResponse which is of type []ArrayItem, an array of type ArrayItem which we are about to declare below.
  • Line 20–23: Declare ArrayItem struct this is the definition of what each item within the array will consist of.
  • Line 27–36: Instantiate arrayItems of type []ArrayItem and populate the values in each index.
  • Line 47: Reference the arrayItems variable in the declaration of the jsonResponse.

Test it out and play with the code here https://play.golang.org/p/YhqKR6lZ_ge.

Consuming JSON

Ok now let’s look at consuming JSON from a rest API, I’m going to mock the returned JSON from the API to simplify the process. Since we have an understanding now of how Go structs relate to JSON strings we’ll jump straight in with a complicated JSON object.

This is the JSON response we are going to get from the API. I’m using the JSON returned from https://httpbin.org/get which is a great website for testing HTTP requests and responses. Here’s the JSON we are going to receive. (Run curl https://httpbin.org/get in your terminal to test)

{
"args": {},
"headers": {
"Accept": "/",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"origin": "83.7.252.17, 83.7.252.17",
"url": "https://httpbin.org/get"
}

Note the “origin” will be different for you as your request will be coming from a different IP. The empty “args” object will contain any arguments passed in the URL. So https://httpbin.org/get?testArg=testValue&x=3would return,

"args": {
"testArg": "testValue",
"x": "3"
}

This args value is a good example of a case when we don’t know what the key name or the values intended type is going to be. To get around this we set “args” type as map[string]interface{} which is a map (a hash table) with keys of type string and values of type interface{}. An empty interface{} is a way of defining a variable in Go as “this could be anything”. At runtime, Go will then allocate the appropriate memory to fit whatever you decide to store in it.

Let’s have a look at an example of using JSON from an API in code and discuss the main sections of the file.

package main

import (
"encoding/json"
"fmt"
)

type JSONResponse struct {
Args map[string]interface{} json:"args"
Headers Headers json:"headers"
Origin string json:"origin"
Url string json:"url"
}

type Headers struct {
Accept string json:"Accept"
Host string json:"Host"
UserAgent string json:"User-Agent"
}

func main() {
jsonString := {"args": {}, "headers": {"Accept": "*/*", "Host": "httpbin.org", "User-Agent": "curl/7.54.0"}, "origin": "89.7.222.10, 89.7.222.10", "url": "https://httpbin.org/get"}

var jsonResponse JSONResponse
err := json.Unmarshal([]byte(jsonString), &jsonResponse)

if err != nil {
	fmt.Println(err)
}

fmt.Println(jsonResponse)

}

  • Line 8–19: Declaration of Structs to match the structure of the returned JSON.
  • Line 22: Mocked JSON response of type string.
  • Line 24: Instantiate jsonResponse of type JSONResponse.
  • Line 25: Unmarshal data from the JSON string and into the jsonResponse variable, the Go tags json:"headers" etc. enable specific JSON key values to be loaded into the appropriate variables within the struct.
  • Line 25: The unmarshal function requires the JSON data is passed in a byte array which is why we cast the JSON string to a byte array.
  • Line 25: The unmarshal function requires a pointer of the variable to be passed in which is why there is an ampersand before jsonResponse. &jsonResponse is the memory location of jsonResponse see here for more information on pointers.
  • Line 27–29: Error check on unmarshalling.
  • Line 31: Printing the jsonResponse Go variable which now contains the data that was in the JSON string.

Try it out for yourself here https://play.golang.org/p/oqFjZii_yjA.

Parsing into an Interface

We now know that we can parse JSON into a map[string]interface{} type if we are unsure of the value types or the key names that are going to be used in the JSON. If we look at the previous example and imagine values were passed in via the URL (https://httpbin.org/get?testArg=testValue&x=3) to result in,

"args": {
"testArg": "testValue",
"x": "3"
}

These values can be accessed simply by referencing jsonResponse.Args["testArg"] and jsonResponse.Args["x"]. Take a look here https://play.golang.org/p/WehVIkK8CRd, I’ve updated the mocked JSON to include the values within the “args” object and added 2 print statements at the end to show you how to access the values.

Hopefully this has helped you gain a better understanding of how to work with JSON in Go. As always any feedback is welcome and please let me know if I’ve made any mistakes.

Thanks for reading

If you liked this post, please do share/like it with all of your programming buddies!

Follow us on Facebook | Twitter

Further reading

JSON Tutorial For Beginners | What is JSON | Learning JSON with JavaScript

The complete beginner’s guide to JSON

Converting JSON to CSV in Java

How to use JSON.stringify() and JSON.parse() in JavaScript

The Complete Guide to JSON Web Tokens

Stateless Authentication with JSON Web Tokens

Parsing JSON in Flutter

Authenticate a Node ES6 API with JSON Web Tokens