【Reprint】Several methods for converting struct to map_string_interface

title: '[Reprint] Several Methods for Converting Struct to map[string]interface{}'
date: 2021-08-09 16:34:33
comment: false
toc: true
category:

Golang
tags:
Reprint
Go
Map
Struct
String
Interface
Methods

This article is a reprint from: Several Methods for Converting Struct to map[string]interface{} | Li Wenzhou's Blog

This article introduces the "pitfalls" you need to know when converting a struct to map[string]interface{} in Go, as well as several methods you need to know.

In Go, we usually use structs to store our data. For example, if we want to store user information, we may define the following struct:

// UserInfo user information
type UserInfo struct {
	Name string `json:"name"`
	Age  int    `json:"age"`
}

u1 := UserInfo{Name: "q1mi", Age: 18}

Suppose we want to convert the above u1 to map[string]interface{}. How do we do it?

Converting Struct to map[string]interface{}#

JSON Serialization#

Isn't this simple? I'll serialize u1 using JSON and then deserialize it into a map. Let's do it:

func main() {
	u1 := UserInfo{Name: "q1mi", Age: 18}

	b, _ := json.Marshal(&u1)
	var m map[string]interface{}
	_ = json.Unmarshal(b, &m)
	for k, v := range m{
		fmt.Printf("key:%v value:%v\n", k, v)
	}
}

Output:

key:name value:q1mi
key:age value:18

It looks fine, but there is actually a "pitfall" here. The json package in Go serializes numeric types (integers, floating-point numbers, etc.) stored in empty interfaces as float64 types.

In other words, m["age"] in the example above is now a float64, not an int. Let's verify this:

func main() {
	u1 := UserInfo{Name: "q1mi", Age: 18}

	b, _ := json.Marshal(&u1)
	var m map[string]interface{}
	_ = json.Unmarshal(b, &m)
	for k, v := range m{
		fmt.Printf("key:%v value:%v value type:%T\n", k, v, v)
	}
}

Output:

key:name value:q1mi value type:string
key:age value:18 value type:float64

Obviously, there is an unexpected behavior here, and we need to find a way to avoid it.

Reflection#

If there is no other way, we need to implement it ourselves. Here, we use reflection to iterate through the fields of the struct and generate a map. The specific code is as follows:

// ToMap converts a struct to a map[string]interface{}
func ToMap(in interface{}, tagName string) (map[string]interface{}, error){
	out := make(map[string]interface{})

	v := reflect.ValueOf(in)
	if v.Kind() == reflect.Ptr {
		v = v.Elem()
	}

	if v.Kind() != reflect.Struct {  // Return an error message for non-structs
		return nil, fmt.Errorf("ToMap only accepts struct or struct pointer; got %T", v)
	}

	t := v.Type()
	// Iterate through the struct fields
	// Use the tagName value as the key in the map; the field value as the value in the map
	for i := 0; i < v.NumField(); i++ {
		fi := t.Field(i)
		if tagValue := fi.Tag.Get(tagName); tagValue != "" {
			out[tagValue] = v.Field(i).Interface()
		}
	}
	return out, nil
}

Let's verify it:

m2, _ := ToMap(&u1, "json")
for k, v := range m2{
	fmt.Printf("key:%v value:%v value type:%T\n", k, v, v)
}

Output:

key:name value:q1mi value type:string
key:age value:18 value type:int

This time, the type of map["age"] is correct.

Third-Party Library: structs#

In addition to implementing it ourselves, there are ready-made solutions on GitHub, such as the third-party library: https://github.com/fatih/structs.

It uses the custom struct tag structs:

// UserInfo user information
type UserInfo struct {
	Name string `json:"name" structs:"name"`
	Age  int    `json:"age" structs:"age"`
}

It is very simple to use:

m3 := structs.Map(&u1)
for k, v := range m3 {
	fmt.Printf("key:%v value:%v value type:%T\n", k, v, v)
}

This structs package also has many other usage examples, you can check the documentation. But please note that this library has been set to read-only by the author.

Converting Nested Struct to map[string]interface{}#

structs itself supports converting nested structs to map[string]interface{}. When encountering a nested struct, it will be converted into a pattern of map[string]interface{} nested with map[string]interface{}.

Let's define a group of nested structs as follows:

// UserInfo user information
type UserInfo struct {
	Name string `json:"name" structs:"name"`
	Age  int    `json:"age" structs:"age"`
	Profile `json:"profile" structs:"profile"`
}

// Profile configuration information
type Profile struct {
	Hobby string `json:"hobby" structs:"hobby"`
}

Declare a struct variable u1:

u1 := UserInfo{Name: "q1mi", Age: 18, Profile: Profile{"Powerball"}}

Third-Party Library: structs#

The code is actually the same as above:

m3 := structs.Map(&u1)
for k, v := range m3 {
	fmt.Printf("key:%v value:%v value type:%T\n", k, v, v)
}

Output:

key:name value:q1mi value type:string
key:age value:18 value type:int
key:profile value:map[hobby:Powerball] value type:map[string]interface {}

From the result, it can be seen that the nested field profile is map[string]interface {}, which is a map nested with a map.

Converting to a Single-Level map Using Reflection#

If we want to convert the nested struct into a single-level map, how can we do it?

We just need to slightly modify the reflection code above:

// ToMap2 converts a struct to a single-level map
func ToMap2(in interface{}, tag string) (map[string]interface{}, error) {

	// This function only accepts struct types
	v := reflect.ValueOf(in)
	if v.Kind() == reflect.Ptr { // Struct pointer
		v = v.Elem()
	}
	if v.Kind() != reflect.Struct {
		return nil, fmt.Errorf("ToMap only accepts struct or struct pointer; got %T", v)
	}

	out := make(map[string]interface{})
	queue := make([]interface{}, 0, 1)
	queue = append(queue, in)

	for len(queue) > 0 {
		v := reflect.ValueOf(queue[0])
		if v.Kind() == reflect.Ptr { // Struct pointer
			v = v.Elem()
		}
		queue = queue[1:]
		t := v.Type()
		for i := 0; i < v.NumField(); i++ {
			vi := v.Field(i)
			if vi.Kind() == reflect.Ptr { // Nested pointer
				vi = vi.Elem()
				if vi.Kind() == reflect.Struct { // Struct
					queue = append(queue, vi.Interface())
				} else {
					ti := t.Field(i)
					if tagValue := ti.Tag.Get(tag); tagValue != "" {
						// Store in the map
						out[tagValue] = vi.Interface()
					}
				}
				break
			}
			if vi.Kind() == reflect.Struct { // Nested struct
				queue = append(queue, vi.Interface())
				break
			}
			// Regular field
			ti := t.Field(i)
			if tagValue := ti.Tag.Get(tag); tagValue != "" {
				// Store in the map
				out[tagValue] = vi.Interface()
			}
		}
	}
	return out, nil
}

Let's test it:

m4, _ := ToMap2(&u1, "json")
for k, v := range m4 {
	fmt.Printf("key:%v value:%v value type:%T\n", k, v, v)
}

Output:

key:name value:q1mi value type:string
key:age value:18 value type:int
key:hobby value:Powerball value type:string

Now we have converted the nested struct into a single-level map. However, please note that in this scenario, we need to avoid duplicate fields between the struct and the nested struct.