Where a Go HTTP server is started, its listener blocks until a client connects to the host + port the listener is bound to. The client connection is started up in its own goroutine, and runs until it returns either success or error. Because goroutines are very small, lightweight stacks, a Go HTTP server can start thousands or even millions of handlers, each in its own goroutine.
The usual practice with HTTP client - server connections is that connections should be short-lived. Ideally, where the client makes a request, if that request cannot be satisfied within a short timeframe, the connection is timed out, an error is returned to the client, the connection is destroyed, and resources are cleaned up on the server. If the client cancels a request, typically that cancellation is honored on the server, the connection is destroyed, and resources are cleaned up on the server.
The usual practice with a running server is that where a shutdown signal is sent to a server, active connections are given the opportunity, via a timer, to complete, before the server, and all related processes, applications, and services, are shut down gracefully. If the shutdown is part of a deploy, an updated server is started, and it handles all new connections. Existing connections to the existing server are permitted to complete or fail, then the existing server is shut down.
Therefore, at both the level of the server, and of each connection, we want to implement timeouts, a timer, and intentionally shut down the server, along with other applications and / or connections to those applications. For example, if our HTTP server is connected to a database, or other services that use persistent connections, we want to close the connections from our server to the database. If our HTTP server is connected to other HTTP services with short-lived connections, there may be nothing at all we have to do, as it is likely those connections will be cleaned up merely by waiting some short time period, and exiting our HTTP server.
The key points in starting, operating, and gracefully shutting down a Go HTTP server are:
The key component in a Go HTTP server is its net.Listener. A net.Listener includes a “listening” or “passive” socket whose job is to “block”, ie wait for the OS to push connections onto the OS’s “accept” queue. Where a Go HTTP server is started, its listener’s Accept method is immediately called and, if there is no data connection on the accept queue, the listener blocks, meaning that the OS puts the Go runtime thread that is running the listener into a paused state. Where a data connection is put onto the accept queue, the kernel wakes up the Go runtime thread, which dequeues the data connection, and passes the data connection to a handler, which the Go runtime starts in its own goroutine.
These completed data connections do more than a listening socket does. They typically pass data back and forth between client and server.
EPoll is the Linux syscall interface that manages sockets as described above. Its name is unfortunate; blocking does not work by polling at intervals. Epoll operates via “interrupt-driven I/O”. Where the state of a socket changes, and where this is a state that the kernel has registered interest in, an interrupt (either a hardware interrupt, in the case of a physical network, or a software interrupt, in the case of a virtual network, unix domain socket, or other non-hardware construct) is sent to the kernel, the kernel looks up the sleeping Go runtime thread that is associated with the registered event on that socket, the Go runtime thread wakes up the handler associated with that registered event / socket, and runs it.
At this point the completed connection is in an “ESTABLISHED” state, and the client may begin to send data over the connection. For a simple “Hello, world!” response, which is likely to be immediately available from the handler, the handler will not block, but will instead immediately send data to the client.
But suppose the handler must do more? Suppose the handler must retrieve data from a database, parse that data, and return it to the client? The same series of steps occurs with this data connection socket as occurred with the listener socket. Where the request is sent to the database, but the database does not respond immediately, the handler will block – ie, the OS kernel will park the Go runtime thread that is waiting on the database’s socket. When the database begins to send data over the connection, the kernel will wake up the Go runtime thread that it parked, the Go runtime thread will wake up the goroutine it assigned to this socket, and the goroutine will run the handler, which will parse the data it received from the database, and send that data to the client. When the handler has completed sending that data back to the client, the handler will close the socket, and eventually the memory represented by the completed goroutine that is running the handler will be garbage collected.
In the below code:
1package server
2
3import (
4 "context"
5 "errors"
6 "github.com/charliemcelfresh/charlie-go/internal/model"
7 "github.com/go-chi/chi/v5"
8 "github.com/go-chi/chi/v5/middleware"
9 "net"
10 "net/http"
11 _ "net/http/pprof"
12 "os/signal"
13 "sync"
14 "syscall"
15 "time"
16)
17
18type Tokener interface {
19 Parse(token string) (Claims, error)
20}
21
22type Logger interface {
23 Error(msg string, args ...any)
24 Info(msg string, args ...any)
25}
26
27type Repo interface {
28 GetItems(ctx context.Context, perPage, page int) ([]model.Item, error)
29 CreateItem(ctx context.Context, item model.Item) (model.Item, error)
30 Close() error
31}
32
33type Server interface {
34 Serve(l net.Listener) error
35 Shutdown(ctx context.Context) error
36}
37
38type Listener interface {
39 Close() error
40 Accept() (net.Conn, error)
41 Addr() net.Addr
42}
43
44type provider struct {
45 jwt Tokener
46 repository Repo
47 logger Logger
48 server Server
49 listener Listener
50}
51
52func NewProvider(repo Repo, listener Listener, logger Logger, token Tokener) provider {
53 provider := provider{
54 jwt: token,
55 repository: repo,
56 logger: logger,
57 }
58
59 router := chi.NewRouter()
60 router.Use(middleware.Logger)
61 router.Use(middleware.Recoverer)
62 router.Use(provider.AddContentTypeToResponse)
63 router.Group(
64 func(r chi.Router) {
65 r.Use(provider.Authorize)
66 r.Get("/api/v1/items", provider.GetItems)
67 r.Post("/api/v1/item", provider.CreateItem)
68 },
69 )
70
71 provider.listener = listener
72
73 httpServer := &http.Server{
74 ReadHeaderTimeout: 5 * time.Second,
75 ReadTimeout: 10 * time.Second,
76 WriteTimeout: 10 * time.Second,
77 IdleTimeout: 10 * time.Second,
78 Handler: router,
79 }
80
81 provider.server = httpServer
82
83 return provider
84}
85
86func (p *provider) Serve(parentCtx context.Context) {
87 // Set up graceful shutdown using os.Signal, via signal.NotifyContext
88 signalCtx, cleanupSignals := signal.NotifyContext(parentCtx, syscall.SIGINT, syscall.SIGTERM)
89 // clean up signal handling when function exits
90 defer cleanupSignals()
91
92 errCh := make(chan error, 1)
93
94 var wg sync.WaitGroup
95 wg.Add(1)
96 go func() {
97 defer wg.Done()
98 if err := p.server.Serve(p.listener); err != nil && !errors.Is(err, http.ErrServerClosed) {
99 p.logger.Error("error starting Server, exiting", "err", err)
100 errCh <- err
101 }
102 }()
103
104 shutdown := func(p *provider) {
105 timeoutCtx, timeoutCancelFunc := context.WithTimeout(context.Background(), 10*time.Second)
106 defer timeoutCancelFunc()
107
108 if err := p.server.Shutdown(timeoutCtx); err != nil {
109 p.logger.Error("httpServer.Shutdown error", "err", err)
110 }
111
112 rErr := p.repository.Close()
113 if rErr != nil {
114 p.logger.Error("error closing p.repository, returning anyway", "rErr", rErr)
115 } else {
116 p.logger.Info("p.repository.Close complete")
117 }
118 p.logger.Info("graceful shutdown complete")
119 }
120
121 select {
122 case <-signalCtx.Done():
123 p.logger.Info("received shutdown signal, exiting", "signalErr", signalCtx.Err())
124 shutdown(p)
125 case err := <-errCh:
126 p.logger.Error("httpServer.Serve encountered error, exiting", "err", err)
127 shutdown(p)
128 }
129 wg.Wait()
130}The above demonstrates one way to start, operate, and gracefully shut down a Go HTTP server. We could start our server in the main goroutine, and put cancellation into a separate goroutine. We could have returned errors to Serve’s calling function. We could avoid allocations at critical performance points, by avoiding the use of Listener and Repo interfaces. We could think about goroutine pools for our handlers. We could add instrumentation both at the network level, and at the code level, in order to uncover any helpful optimizations.
The point is that the Go http package provides, out of the box, a high performance HTTP server, that adheres to typical HTTP practices, and uses the power of Go and also of the OS.