LDF: Format Benchmarks

Comparison:

ldf-stable: LDF with stable option enabled
buffer: LuaJIT's String Buffers
bitser
binser
msgpack

Notes:

5 measures are taken and aggregated (average) for each benchmark.
Each measure is one invocation of a benchmark, which will perform packing/unpacking of the data structure in a loop until a certain amount of time has passed.
Lower is better.
Row values are percentages of the raw value. These values are not meant to be compared between rows, because the percentage is relative to the row.
I'm still not sure if the way the benchmarks are aggregated (using a geometric mean) is sound, but it seems to be.
There are other metrics than performances. For example, bitser doesn't deal with endianness or with double NaN tagging issues.
The size metric is mostly relevant if no general purpose byte compression is applied on the data afterwards, as it can be the case with small network messages.

Results (JIT on)

job	command
array-16	`luajit benchmarks/structs.lua <impl> array 16`
array-32	`luajit benchmarks/structs.lua <impl> array 32`
array-128	`luajit benchmarks/structs.lua <impl> array 128`
array-1024	`luajit benchmarks/structs.lua <impl> array 1024`
array-1e6	`luajit benchmarks/structs.lua <impl> array 1000000`
farray-16	`luajit benchmarks/structs.lua <impl> farray 16`
farray-1e3	`luajit benchmarks/structs.lua <impl> farray 1000`
farray-1e6	`luajit benchmarks/structs.lua <impl> farray 1000000`
vectors-3-1000	`luajit benchmarks/structs.lua <impl> vectors 3 1000`
entities-1	`luajit benchmarks/structs.lua <impl> entities 1`
entities-100	`luajit benchmarks/structs.lua <impl> entities 100`
entities-1000	`luajit benchmarks/structs.lua <impl> entities 1000`
tree-2-8	`luajit benchmarks/structs.lua <impl> tree 2 8`
tree-8-7	`luajit benchmarks/structs.lua <impl> tree 8 7`

Packing time (ns, cpu time)

	buffer	ldf	ldf-stable	msgpack	bitser	binser	raw value
array-16	6	44	46	76	28	100	1470
array-32	8	37	37	81	30	100	2158
array-128	9	44	42	75	37	100	6427
array-1024	5	31	28	93	28	100	88983
array-1e6	1	9	8	95	7	100	374161000
farray-16	4	49	54	89	40	100	2268
farray-1e3	3	19	20	96	39	100	128875
farray-1e6	1	8	7	100	13	97	393878200
vectors-3-1000	3	32	44	90	45	100	694253
entities-1	2	21	100	20	20	49	4892
entities-100	2	9	100	15	7	27	393394
entities-1000	2	10	100	28	9	39	3610644
tree-2-8	4	45	63	36	40	100	94372
tree-8-7	7	50	69	63	48	100	214977400
geo. mean	3	24	40	59	23	81

Unpacking time (ns, cpu time)

	buffer	ldf	ldf-stable	msgpack	bitser	binser	raw value
array-16	6	26	25	100	17	21	3201
array-32	5	18	18	100	13	15	5816
array-128	4	16	16	100	13	9	19915
array-1024	11	74	74	87	100	30	43242
array-1e6	13	81	84	74	100	81	48158350
farray-16	15	85	83	91	100	100	1226
farray-1e3	12	47	47	58	100	76	39927
farray-1e6	15	49	50	58	100	77	40412160
vectors-3-1000	20	80	83	100	85	76	420158
entities-1	10	36	36	100	28	30	3369
entities-100	9	20	24	100	18	18	268912
entities-1000	14	24	25	100	22	19	2338917
tree-2-8	12	43	43	100	42	34	121626
tree-8-7	7	26	29	100	24	28	529984000
geo. mean	10	38	39	89	40	34

Size (bytes)

	buffer	ldf	ldf-stable	msgpack	bitser	binser	raw value
array-16	100	12	12	13	13	13	146
array-32	100	12	12	12	12	12	290
array-128	100	17	17	11	16	14	1154
array-1024	100	30	30	29	31	21	9219
array-1e6	100	54	54	54	55	99	9000006
farray-16	100	95	95	95	95	95	146
farray-1e3	100	100	100	100	100	100	9003
farray-1e6	100	100	100	100	100	100	9000006
vectors-3-1000	100	64	64	64	72	67	29003
entities-1	100	44	44	44	51	61	61
entities-100	100	22	22	47	23	32	6100
entities-1000	100	26	26	53	30	35	61001
tree-2-8	100	18	18	18	36	36	2814
tree-8-7	100	12	12	12	15	15	19473554
geo. mean	100	32	32	35	36	38

Environment

LuaJIT

LuaJIT 2.1.1723675123 -- Copyright (C) 2005-2023 Mike Pall. https://luajit.org/

CPU

Architecture:             x86_64
  CPU op-mode(s):         32-bit, 64-bit
  Address sizes:          39 bits physical, 48 bits virtual
  Byte Order:             Little Endian
CPU(s):                   4
  On-line CPU(s) list:    0-3
Vendor ID:                GenuineIntel
  Model name:             Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz
    CPU family:           6
    Model:                94
    Thread(s) per core:   1
    Core(s) per socket:   4
    Socket(s):            1
    Stepping:             3
    CPU max MHz:          3600.0000
    CPU min MHz:          800.0000
    BogoMIPS:             6402.62
Caches (sum of all):
  L1d:                    128 KiB (4 instances)
  L1i:                    128 KiB (4 instances)
  L2:                     1 MiB (4 instances)
  L3:                     6 MiB (1 instance)
NUMA:
  NUMA node(s):           1
  NUMA node0 CPU(s):      0-3