STRLCPY/graphql-engine

server: Use mimalloc in executables

This fixes the poor memory behavior we observe in #9447. Observe a
haskell program has layers of memory usage/management, from inner to
outer:

1) objects on the haskell heap aka "live data"
2) blocks of memory managed by the haskell RTS, containing live data
   (and including overheads from fragmentation and space for
   copying-into during GC)
3) foreign data malloc'd and free'd from haskell, but indirectly (during
   GC, "finalizers" are run that call `free` generally)
4) the implementation of malloc itself we're linked against maintains a
   list of blocks of free memory  requested from the OS; it decides when
   to return blocks back (reflected in lower RSS from top), and
   fragmentation is also a concern here
   4a) should the malloc decide to return memory, it might use MADV_FREE
       or another mechanism which won't be reflected in RSS unless there is
       memory pressure. This further complicates things

(1) and (2) can be monitored from /dev/rts_stats . (3) can be monitored
with heaptrack, valgrind, etc. (4) was where our issues were here.

mimalloc helps because:

- it seems to handle fragmentation better, for the large response sizes
  in our reproo
- it is more eager to return memory back to the OS (but note: newer
  versions use MADV_FREE and probably aren't useable for us. See:
  https://github.com/microsoft/mimalloc/issues/776 )

note that `static.o` gets linked second, but according to my tests the
order shouldn't matter (despite what mimalloc docs suggest):

```
gcc '-fuse-ld=lld' -Wl,--no-as-needed -o
/home/me/Work/hasura/graphql-engine-mono/dist-newstyle/build/x86_64-linux/ghc-9.4.5/graphql-engine-1.0.0/x/graphql-engine/opt/build/graphql-engine/graphql-engine
-lm -no-pie -Wl,--gc-sections
/home/me/Work/hasura/graphql-engine-mono/dist-newstyle/build/x86_64-linux/ghc-9.4.5/graphql-engine-1.0.0/x/graphql-engine/opt/build/graphql-engine/graphql-engine-tmp/Main.o
/home/me/Work/hasura/graphql-engine-mono/dist-newstyle/build/x86_64-linux/ghc-9.4.5/graphql-engine-1.0.0/x/graphql-engine/opt/build/graphql-engine/graphql-engine-tmp/../preload-mimalloc/mimalloc/src/static.o
…
```

NOTE: the promptness of memory reclamation depends on both
HASURA_GRAPHQL_PG_TIMEOUT and HASURA_GRAPHQL_PG_CONN_LIFETIME. By
default a connection's resources will be GC'd after 3min when idle, or
after no more than 10 minutes.
GitOrigin-RevId: 9b522c39159b4c710c5672cc8c62c5c723d4bd13

Brandon Simmons committed with hasura-bot 2 months ago

cde5efae

1 parent 9c2ea26c

Total 4 files Show one by one

■ ■ ■ ■ ■ ■

cabal.project

		skipped 30 lines
31	31		-- For tooling, e.g. 'weeder', and IDE-like stuff:
32	32		ghc-options: -fwrite-ide-info
33	33
	34	+	-- we statically link malloc from mimalloc. Out of an abundance of caution,
	35	+	-- disable special treatment of these in all the foreign code we build. The
	36	+	-- only risk is potential for some missed optimizations.
	37	+	-- See: https://github.com/microsoft/mimalloc/issues/785
	38	+	-- NOTE: cc-options is not recognized here for some reason, so we use ghc-options+optc:
	39	+	ghc-options: -optc-fno-builtin-malloc -optc-fno-builtin-calloc -optc-fno-builtin-realloc -optc-fno-builtin-free
	40	+
34	41		haddock-html: true
35	42		haddock-hoogle: true
36	43		haddock-hyperlink-source: true
		skipped 76 lines

■ ■ ■ ■ ■ ■

preload-mimalloc/README.md

1	+	# Vendored mimalloc
2	+
3	+
4	+	This is [mimalloc](https://github.com/microsoft/mimalloc) vendored for use in
5	+	our server code. You can upgrade mimalloc by editing `VERSION` in
6	+	`update-sources.sh` and re-running it.
7	+
8	+	Usage sites in haskell projects look like:
9	+
10	+	-- See 'preload-mimalloc/README.md'
11	+	-- We want this in all server binary executable sections
12	+	common preload-mimalloc
13	+	c-sources: ../preload-mimalloc/mimalloc/src/static.c
14	+	include-dirs: ../preload-mimalloc/mimalloc/include
15	+	cc-options: -DMI_MALLOC_OVERRIDE
16	+
17	+	## Usage in haskell projects
18	+
19	+	In [#9447](https://github.com/hasura/graphql-engine-mono/pull/9447) we see
20	+	overloading the server with large and slow queries results in high memory usage
21	+	and what looks like a space leak, involving foreign data from libpq.
22	+
23	+	See that ticket for details, but mimalloc helps (relative to my glibc) by:
24	+
25	+	- better avoiding fragmentation, resulting in lower overall memory usage (4GB peak 2.8 aftervs.
26	+	- eagerly decomitting memory so that OS-reported memory usage is a useful
27	+	metric (we do the same in the haskell RTS by specifying
28	+	`--disable-delayed-os-memory-return`)
29	+
30	+	We also benefit by standardizing this, rather than relying on users' glibc
31	+	version, which might vary in behavior, making debugging difficult.
32	+
33	+	Other implementations such as tcmalloc or jemalloc were not tested but might
34	+	perform better or be more tunable.
35	+
36	+	### Why not LD_PRELOAD?
37	+
38	+	It's most common and simplest to use the dynamic linker to override malloc.
39	+	This would work in docker containers we ship, but we want all users to have the
40	+	same (better) behavior out of the box.
41	+

■ ■ ■ ■ ■ ■

preload-mimalloc/update-sources.sh

1	+	#!/usr/bin/env bash
2	+	set -euo pipefail
3	+
4	+	# You can use this to update the mimalloc source code vendored here, if you
5	+	# wish to upgrade the version for instance. This should be idempotent.
6	+	#
7	+	# NOTE!: v2.1.2 regresses from our point of view. See:
8	+	# https://github.com/microsoft/mimalloc/issues/776
9	+	VERSION=v2.1.1
10	+
11	+
12	+	THIS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" # ... https://stackoverflow.com/a/246128/176841
13	+	cd "$THIS_DIR"
14	+
15	+	rm -rf mimalloc
16	+	git clone [email protected]:microsoft/mimalloc.git 2>/dev/null
17	+	(
18	+	cd mimalloc
19	+	git checkout "$VERSION" 2>/dev/null
20	+	# stuff we don't need:
21	+	rm -rf .git docs doc cmake bin azure-pipelines.yml CMakeLists.txt .gitattributes .gitignore ide mimalloc.pc.in test
22	+	) # in mimalloc/
23	+
24	+	echo "Done. $VERSION vendored in 'mimalloc/'"
25	+

■ ■ ■ ■ ■ ■

server/graphql-engine.cabal

		skipped 133 lines
134	134		-- workaround for limitations in monitoring tools than anything...
135	135		"-with-rtsopts=-N -I0 -T -kc8K --disable-delayed-os-memory-return"
136	136
	137	+
	138	+	-- See 'preload-mimalloc/README.md'
	139	+	-- We want this in all server binary executable sections
	140	+	common preload-mimalloc
	141	+	c-sources: ../preload-mimalloc/mimalloc/src/static.c
	142	+	include-dirs: ../preload-mimalloc/mimalloc/include
	143	+	cc-options: -DMI_MALLOC_OVERRIDE
	144	+	cc-options: -DMI_STATIC_LIB
	145	+	cc-options: -O3 -DNDEBUG -fPIC -Wall -Wextra -Wno-unknown-pragmas -fvisibility=hidden -Wstrict-prototypes -ftls-model=initial-exec -std=gnu11
	146	+
137	147		common lib-depends
138	148		build-depends: Spock-core
139	149		, aeson
		skipped 733 lines
873	883
874	884		executable graphql-engine
875	885		import: common-all, common-exe
	886	+	import: preload-mimalloc
876	887
877	888		if flag(optimize-hasura)
878	889		ghc-options:
		skipped 471 lines