Go在探探后端的工程实践

1 .Engineering Practices in Tantan using Golang Henry Ren 

2 . April 15 2018 Engineering Practices in Tantan using Golang Henry Ren Product Backend Team Lead 探探科技 henry@tantanapp.com GopherChina2018 

3 .Agenda ● Overview of Tantan ● Go in Tantan Backend ● Engineering Practices ○ Daily Development ○ Testing Go Code with ○ Building RESTful API ● Architecture Evolvement ● Conclusion GopherChina2018 

4 .Overview of Tantan GopherChina2018 

5 .#1 How Tantan works GopherChina2018 

6 .#2 Daily Swipes 1+ Billion RESTful API PUT /users/me/relationships/:uid GopherChina2018 

7 .Go in Tantan Backend GopherChina2018 

8 .#1 Building Tantan Backend ● Goals ○ Clean Code Simplicity, Readability, Maintainability, Testability ○ Development Efficiency Ease of Development & Deployment ○ Scalability Vertical & Horizontal ○ Performant ● Challenges ○ Complexities Product Requirements ○ Software Quality ○ Continuous Iterations Development & Deployment ○ Small Team 3 Backend Developers in the Beginning GopherChina2018 

9 .#2 Tech Stacks ● Team members in May, 2014 ○ Developers with different background (PHP, C#, Python, C) ○ Developers speak different languages (English, Finnish, Swedish, Chinese) ○ Build first version within 4-5 weeks ● Tech stacks ○ Golang ■ Simplicity ■ Performant ■ Fun ○ PostgreSQL ■ PostGiS Extension for Location Based Services ■ Advanced Data Types, Partial Indexes, Stored Procedures etc. GopherChina2018 

10 .#3 Go versions 2014/06/18 2015/08/19 2016/08/15 2017/08/24 Go1.2 Go1.3 Go1.4 Go1.5 Go1.6 Go1.7 Go1.8 Go1.9 Go1.10 2013/12/01 2014/12/10 2016/02/17 2017/02/16 2018/02/16 GopherChina2018 

11 .#4 Lines of Go Code Lines of Go code in Backend Services Repository GopherChina2018 

12 .#5 Backend Services Built in Go ● HTTP web servers ○ Using net/http package ○ RESTful API Services ○ Media (Image/Video/Audio) Uploading & Downloading Service ● RPC servers ○ Using net/rpc package ● Web apps ○ Using Revel web framework ● Cli programs ● Cron jobs ● etc. GopherChina2018 

13 .#6 Engineering Practices ● Daily Development ● Testing Go Code with ● Building RESTful API GopherChina2018 

14 .Engineering Practices: Daily Development GopherChina2018 

15 .#1 Repository ● Single repository using Github ● Repository organization ○ cmd/: all main packages or files ○ app/: specific app implementations ○ version/: binary versioning ○ vendor/: vendoring packages ○ doc/: documentation in Markdown ○ db/: database changes ○ etc. ● Workflow ○ Fork -> Pull Request -> Code Review GopherChina2018 

16 .#2 Go Tools ● go fmt ● go test ● go build / go install / go run ○ “It's a fast, statically typed, compiled language that feels like a dynamically typed, interpreted language.” ● go doc ○ go doc json / go doc json.Decoder / go doc json.Decoder.Decode ○ godoc -http=:8080 // check go documentation and standard libraries while golang.org not available ● go vet ○ report likely mistakes in packages ● Complete list: https://golang.org/cmd/ GopherChina2018 

17 .#3 IDEs ● Vim, Emacs, Sublime Text, GoLand, VS Code, etc. ● Development Plugins with different Go tools integrated (vscode-go for example) ○ Completion Lists (using gocode) ○ Build-on-save (using go build and go test) ○ Lint-on-save (using golint or gometalinter) ○ Format on save as well as format manually (using goreturns or goimports or gofmt) ○ Add Imports (using gopkgs) ○ Add/Remove Tags on struct fields (using gomodifytags) ○ Run Tests under the cursor, in current file, in current package, in the whole workspace (using go test) ○ Show code coverage ○ etc. GopherChina2018 

18 .#4 Packages ● Standard Packages ○ net/http ○ encoding/json ○ context cancellation, timeouts, request scoped data etc ○ reflect ○ etc. ● External Packages ○ github.com/julienschmidt/httprouter ○ github.com/braintree/manners ○ github.com/revel/revel ○ github.com/stretchr/testify/assert ○ etc. go list -f '{{ join .Imports "\n" }}' ./... GopherChina2018 

19 .#5 Versioning ● Compile version info into Go binary ○ go build -o tantan-swipe-service \ ○ -ldflags "-X tantan/version.version=master-21a5f142fe3041a1ef6ec17d86a15423829d5ddc \ ○ -X tantan/version.date=2018-04-05T20:44:34+0800" \ ○ cmd/tantan-swipe-service/main.go ○ ○ ./tantan-swipe-service -version -X importpath.name=value ○ Version: master-21a5f142fe3041a1ef6ec17d86a15423829d5ddc Set the value of the string variable ○ Binary: ./tantan-swipe-service in importpath named name to value. ○ Compile date: 2018-04-05T20:44:34+0800 https://golang.org/cmd/link/ GopherChina2018 

20 .package version import ( "flag" "fmt" "io" "os" ) var showVersion = flag.Bool("version", false, "Print version of this binary") var ( version string date string ) GopherChina2018 

21 .func init() { if !flag.Parsed() { flag.Parse() } if showVersion != nil && *showVersion { printVersion(os.Stdout, version, date) os.Exit(0) } } func printVersion(w io.Writer, version string, date string) { fmt.Fprintf(w, "Version: %s\n", version) fmt.Fprintf(w, "Binary: %s\n", os.Args[0]) fmt.Fprintf(w, "Compile date: %s\n", date) } GopherChina2018 

22 .#6 Profiling ● Profiling Types ○ cpu profile ○ mem profile ○ blocking profile, goroutine profile etc. ● Profiling Generation ○ runtime/pprof ○ net/http/pprof ○ go test . -bench . -cpuprofile prof.cpu ● Profiling Visualization ○ go tool pprof cpu.prof ○ Flame graph profiler for Go programs: https://github.com/uber/go-torch GopherChina2018 

23 .#7 Code Review ● Coding standards ● Go Code Review Comments https://github.com/golang/go/wiki/CodeReviewComments GopherChina2018 

24 .#8 Others ● Common issues ○ for … range loop variable reuse ○ http response body not closed ○ goroutine lifetime ○ etc. ● Concurrency patterns ○ goroutine: execution ○ channel: communication message queuing ○ select: coordination ● Reflection: runtime reflection, allows a program to manipulate objects with arbitrary types ● Monitoring: Integration with Prometheus GopherChina2018 

25 .Engineering Practices: Testing Go Code GopherChina2018 

26 .http://www.osnews.com/story/19266/WTFs_m GopherChina2018 

27 .#1 Testable Code ● Why it matters? ○ Development Time vs. Maintenance Time ○ Make iterations or CI/CD easier (bug fixes etc.) ● Code Quality ○ Readability ○ Maintainability ○ Testability ● Continuous Integration ○ Automatic Testing GopherChina2018 

28 .#2 Testing in Go ● A lightweight test framework using: ○ go test command ○ testing package ● Go test files: _test.go // foo_test.go (excluded from go build) import "testing" func TestXxx(t *testing.T) {} https://martinfowler.com/bliki/UnitTest.html func ExampleXxx() {} func BenchmarkXxx(b *testing.B) {} GopherChina2018 

29 .testing.TB interface ● Implemented by testing.T and testing.B structs type TB interface { Error(args ...interface{}) Errorf(format string, args ...interface{}) Fail() FailNow() // signal failures Failed() bool Fatal(args ...interface{}) // … } GopherChina2018