Extremely fast, light weight, zero alloc logfmt logging library for Go.

  • By Zerodha Tech
  • Last update: Jan 2, 2023
  • Comments: 8

💥 logf

Go Reference Go Report Card GitHub Actions

logf is a high-performance, zero-alloc logging library for Go applications with a minimal API overhead. It's also the fastest logfmt logging library for Go.

logf emits structured logs in logfmt style. logfmt is a flexible format that involves key=value pairs to emit structured log lines. logfmt achieves the goal of generating logs that are not just machine-friendly but also readable by humans, unlike the clunky JSON lines.

Example

package main

import (
	"time"

	"github.com/zerodha/logf"
)

func main() {
	logger := logf.New(logf.Opts{
		EnableColor:          true,
		Level:                logf.DebugLevel,
		CallerSkipFrameCount: 3,
		EnableCaller:         true,
		TimestampFormat:      time.RFC3339Nano,
		DefaultFields:        []any{"scope", "example"},
	})

	// Basic logs.
	logger.Info("starting app")
	logger.Debug("meant for debugging app")

	// Add extra keys to the log.
	logger.Info("logging with some extra metadata", "component", "api", "user", "karan")

	// Log with error key.
	logger.Error("error fetching details", "error", "this is a dummy error")

	// Log the error and set exit code as 1.
	logger.Fatal("goodbye world")
}

Text Output

timestamp=2022-07-07T12:09:10.221+05:30 level=info message="starting app"
timestamp=2022-07-07T12:09:10.221+05:30 level=info message="logging with some extra metadata" component=api user=karan
timestamp=2022-07-07T12:09:10.221+05:30 level=error message="error fetching details" error="this is a dummy error"
timestamp=2022-07-07T12:09:10.221+05:30 level=fatal message="goodbye world"

Console Output

Why another lib

There are several logging libraries, but the available options didn't meet our use case.

logf meets our constraints of:

  • Clean API
  • Minimal dependencies
  • Structured logging but human-readable (logfmt!)
  • Sane defaults out of the box

Benchmarks

You can run benchmarks with make bench.

No Colors (Default)

BenchmarkNoField-8                       7884771               144.2 ns/op             0 B/op          0 allocs/op
BenchmarkOneField-8                      6251565               186.7 ns/op             0 B/op          0 allocs/op
BenchmarkThreeFields-8                   6273717               188.2 ns/op             0 B/op          0 allocs/op
BenchmarkErrorField-8                    6687260               174.8 ns/op             0 B/op          0 allocs/op
BenchmarkHugePayload-8                   3395139               360.3 ns/op             0 B/op          0 allocs/op
BenchmarkThreeFields_WithCaller-8        2764860               437.9 ns/op           216 B/op          2 allocs/op

With Colors

BenchmarkNoField_WithColor-8             6501867               186.6 ns/op             0 B/op          0 allocs/op
BenchmarkOneField_WithColor-8            5938155               205.7 ns/op             0 B/op          0 allocs/op
BenchmarkThreeFields_WithColor-8         4613145               379.4 ns/op             0 B/op          0 allocs/op
BenchmarkErrorField_WithColor-8          3512522               353.6 ns/op             0 B/op          0 allocs/op
BenchmarkHugePayload_WithColor-8         1520659               799.5 ns/op             0 B/op          0 allocs/op

For a comparison with existing popular libs, visit uber-go/zap#performance.

LICENSE

LICENSE

Download

logf.zip

Comments(8)

  • 1

    fix: remove pointer receiver

    Bench stat:

    name                      old time/op    new time/op    delta
    NoField-8                    179ns ± 3%     114ns ± 3%  -36.06%  (p=0.000 n=3+3)
    OneField-8                   283ns ± 1%     244ns ± 3%  -13.69%  (p=0.009 n=3+3)
    ThreeFields-8                338ns ± 2%     288ns ± 1%  -14.69%  (p=0.002 n=3+3)
    ErrorField-8                 381ns ±13%     267ns ± 1%  -29.87%  (p=0.047 n=3+3)
    HugePayload-8               1.11µs ± 6%    0.95µs ±21%     ~     (p=0.239 n=3+3)
    ThreeFields_WithCaller-8     932ns ± 4%     405ns ± 2%  -56.57%  (p=0.001 n=3+3)
    NoField_WithColor-8          320ns ± 1%     170ns ± 2%  -46.97%  (p=0.000 n=3+3)
    OneField_WithColor-8         441ns ± 9%     323ns ± 3%  -26.79%  (p=0.023 n=3+3)
    ThreeFields_WithColor-8      569ns ±19%     388ns ± 2%     ~     (p=0.081 n=3+3)
    ErrorField_WithColor-8       500ns ±10%     350ns ± 2%  -30.00%  (p=0.027 n=3+3)
    HugePayload_WithColor-8     1.24µs ± 7%    1.03µs ± 1%  -16.95%  (p=0.036 n=3+3)
    
    name                      old alloc/op   new alloc/op   delta
    NoField-8                    0.00B          0.00B          ~     (zero variance)
    OneField-8                    336B ± 0%      336B ± 0%     ~     (zero variance)
    ThreeFields-8                 336B ± 0%      336B ± 0%     ~     (zero variance)
    ErrorField-8                  368B ± 0%      352B ± 0%     ~     (zero variance)
    HugePayload-8                 919B ± 0%      919B ± 0%     ~     (zero variance)
    ThreeFields_WithCaller-8      656B ± 0%      336B ± 0%     ~     (zero variance)
    NoField_WithColor-8          0.00B          0.00B          ~     (zero variance)
    OneField_WithColor-8          336B ± 0%      336B ± 0%     ~     (zero variance)
    ThreeFields_WithColor-8       336B ± 0%      336B ± 0%     ~     (zero variance)
    ErrorField_WithColor-8        368B ± 0%      352B ± 0%     ~     (zero variance)
    HugePayload_WithColor-8       919B ± 0%      918B ± 0%     ~     (zero variance)
    
    name                      old allocs/op  new allocs/op  delta
    NoField-8                     0.00           0.00          ~     (zero variance)
    OneField-8                    2.00 ± 0%      2.00 ± 0%     ~     (zero variance)
    ThreeFields-8                 2.00 ± 0%      2.00 ± 0%     ~     (zero variance)
    ErrorField-8                  4.00 ± 0%      3.00 ± 0%     ~     (zero variance)
    HugePayload-8                 3.00 ± 0%      3.00 ± 0%     ~     (zero variance)
    ThreeFields_WithCaller-8      7.00 ± 0%      2.00 ± 0%     ~     (zero variance)
    NoField_WithColor-8           0.00           0.00          ~     (zero variance)
    OneField_WithColor-8          2.00 ± 0%      2.00 ± 0%     ~     (zero variance)
    ThreeFields_WithColor-8       2.00 ± 0%      2.00 ± 0%     ~     (zero variance)
    ErrorField_WithColor-8        4.00 ± 0%      3.00 ± 0%     ~     (zero variance)
    HugePayload_WithColor-8       3.00 ± 0%      3.00 ± 0%     ~     (zero variance)
    
  • 2

    Infof helpers

    Hi,

    logf looks awesome. What do you think to provide a Infof(format, args...) variant ? I known this API won't allow to add fields, but that may be handy to format message.

    Regards,

  • 3

    draft: new field input format

    Benchstat:

    name                      old time/op    new time/op    delta
    NoField-8                    179ns ± 3%     182ns ± 7%     ~     (p=0.687 n=3+3)
    OneField-8                   283ns ± 1%     214ns ± 5%  -24.39%  (p=0.008 n=3+3)
    ThreeFields-8                338ns ± 2%     268ns ± 1%  -20.82%  (p=0.002 n=3+3)
    ErrorField-8                 381ns ±13%     270ns ±11%  -29.14%  (p=0.029 n=3+3)
    HugePayload-8               1.11µs ± 6%    0.69µs ± 2%  -37.92%  (p=0.004 n=3+3)
    ThreeFields_WithCaller-8     932ns ± 4%     860ns ±13%     ~     (p=0.337 n=3+3)
    NoField_WithColor-8          320ns ± 1%     298ns ±10%     ~     (p=0.316 n=3+3)
    OneField_WithColor-8         441ns ± 9%     351ns ± 7%  -20.39%  (p=0.026 n=3+3)
    ThreeFields_WithColor-8      569ns ±19%     419ns ± 1%     ~     (p=0.112 n=3+3)
    ErrorField_WithColor-8       500ns ±10%     373ns ± 2%  -25.36%  (p=0.038 n=3+3)
    HugePayload_WithColor-8     1.24µs ± 7%    0.88µs ± 8%  -28.93%  (p=0.003 n=3+3)
    
    name                      old alloc/op   new alloc/op   delta
    NoField-8                    0.00B          0.00B          ~     (zero variance)
    OneField-8                    336B ± 0%       32B ± 0%     ~     (zero variance)
    ThreeFields-8                 336B ± 0%       96B ± 0%     ~     (zero variance)
    ErrorField-8                  368B ± 0%       80B ± 0%     ~     (zero variance)
    HugePayload-8                 919B ± 0%      320B ± 0%     ~     (zero variance)
    ThreeFields_WithCaller-8      656B ± 0%      416B ± 0%     ~     (zero variance)
    NoField_WithColor-8          0.00B          0.00B          ~     (zero variance)
    OneField_WithColor-8          336B ± 0%       32B ± 0%     ~     (zero variance)
    ThreeFields_WithColor-8       336B ± 0%       96B ± 0%     ~     (zero variance)
    ErrorField_WithColor-8        368B ± 0%       80B ± 0%     ~     (zero variance)
    HugePayload_WithColor-8       919B ± 0%      320B ± 0%     ~     (zero variance)
    
    name                      old allocs/op  new allocs/op  delta
    NoField-8                     0.00           0.00          ~     (zero variance)
    OneField-8                    2.00 ± 0%      1.00 ± 0%     ~     (zero variance)
    ThreeFields-8                 2.00 ± 0%      1.00 ± 0%     ~     (zero variance)
    ErrorField-8                  4.00 ± 0%      3.00 ± 0%     ~     (zero variance)
    HugePayload-8                 3.00 ± 0%      1.00 ± 0%     ~     (zero variance)
    ThreeFields_WithCaller-8      7.00 ± 0%      6.00 ± 0%     ~     (zero variance)
    NoField_WithColor-8           0.00           0.00          ~     (zero variance)
    OneField_WithColor-8          2.00 ± 0%      1.00 ± 0%     ~     (zero variance)
    ThreeFields_WithColor-8       2.00 ± 0%      1.00 ± 0%     ~     (zero variance)
    ErrorField_WithColor-8        4.00 ± 0%      3.00 ± 0%     ~     (zero variance)
    HugePayload_WithColor-8       3.00 ± 0%      1.00 ± 0%     ~     (zero variance)
    
  • 4

    feat: Improve performance by writing directly to a buffer.

    This also removes intermediary map & slices and writes directly to the buffer.

    Benchmarks before this PR:

    2022-06-27_16-04

    Benchmarks after this PR:

    2022-06-27_16-03

    This makes this atleast 4x faster.

  • 5

    feat: Change any to interface{} to support Go 1.17.

    Currently logf fails to compile on Go 1.17 due to using any in place of interface{}. I think there's no technical reason to not support 1.17 until it reaches end of life.

    We can change this to any once 1.17 reaches end of life.

  • 6

    fix: more tests, improve codecov, broken caller

    Improves code coverage and adds + fixes while writing a few tests.

    go test -cover ./...
    ok      github.com/zerodha/logf 0.076s  coverage: 93.5% of statements
    ?       github.com/zerodha/logf/examples        [no test files]
    
  • 7

    fix: avoid alloc using strings.IndexFunc instead

    Uses strings.IndexFunc to avoid allocs.

    After patching: post_fix_1

    Before patching:

    pre_fix_1

    Benchstat:

    name                      old time/op    new time/op    delta
    NoField-8                    179ns ± 3%     176ns ± 3%     ~     (p=0.567 n=3+3)
    OneField-8                   283ns ± 1%     306ns ± 4%     ~     (p=0.064 n=3+3)
    ThreeFields-8                338ns ± 2%     373ns ± 5%  +10.41%  (p=0.044 n=3+3)
    ErrorField-8                 381ns ±13%     345ns ± 3%     ~     (p=0.291 n=3+3)
    HugePayload-8               1.11µs ± 6%    1.16µs ±22%     ~     (p=0.745 n=3+3)
    ThreeFields_WithCaller-8     932ns ± 4%    1043ns ±14%     ~     (p=0.287 n=3+3)
    NoField_WithColor-8          320ns ± 1%     320ns ±11%     ~     (p=0.978 n=3+3)
    OneField_WithColor-8         441ns ± 9%     421ns ± 2%     ~     (p=0.414 n=3+3)
    ThreeFields_WithColor-8      569ns ±19%     519ns ± 2%     ~     (p=0.456 n=3+3)
    ErrorField_WithColor-8       500ns ±10%     465ns ± 2%     ~     (p=0.317 n=3+3)
    HugePayload_WithColor-8     1.24µs ± 7%    1.38µs ±22%     ~     (p=0.450 n=3+3)
    
    name                      old alloc/op   new alloc/op   delta
    NoField-8                    0.00B          0.00B          ~     (zero variance)
    OneField-8                    336B ± 0%      336B ± 0%     ~     (zero variance)
    ThreeFields-8                 336B ± 0%      336B ± 0%     ~     (zero variance)
    ErrorField-8                  368B ± 0%      368B ± 0%     ~     (zero variance)
    HugePayload-8                 919B ± 0%      919B ± 0%     ~     (zero variance)
    ThreeFields_WithCaller-8      656B ± 0%      656B ± 0%     ~     (zero variance)
    NoField_WithColor-8          0.00B          0.00B          ~     (zero variance)
    OneField_WithColor-8          336B ± 0%      336B ± 0%     ~     (zero variance)
    ThreeFields_WithColor-8       336B ± 0%      336B ± 0%     ~     (zero variance)
    ErrorField_WithColor-8        368B ± 0%      368B ± 0%     ~     (zero variance)
    HugePayload_WithColor-8       919B ± 0%      919B ± 0%     ~     (p=0.423 n=3+3)
    
    name                      old allocs/op  new allocs/op  delta
    NoField-8                     0.00           0.00          ~     (zero variance)
    OneField-8                    2.00 ± 0%      2.00 ± 0%     ~     (zero variance)
    ThreeFields-8                 2.00 ± 0%      2.00 ± 0%     ~     (zero variance)
    ErrorField-8                  4.00 ± 0%      4.00 ± 0%     ~     (zero variance)
    HugePayload-8                 3.00 ± 0%      3.00 ± 0%     ~     (zero variance)
    ThreeFields_WithCaller-8      7.00 ± 0%      7.00 ± 0%     ~     (zero variance)
    NoField_WithColor-8           0.00           0.00          ~     (zero variance)
    OneField_WithColor-8          2.00 ± 0%      2.00 ± 0%     ~     (zero variance)
    ThreeFields_WithColor-8       2.00 ± 0%      2.00 ± 0%     ~     (zero variance)
    ErrorField_WithColor-8        4.00 ± 0%      4.00 ± 0%     ~     (zero variance)
    HugePayload_WithColor-8       3.00 ± 0%      3.00 ± 0%     ~     (zero variance)
    
  • 8

    More performance fixes

    • Remove a single buffer from logger and use a buffer pool.
    • Add a new logger called field logger which contains it's own fields, so it's not necessary to lock the whole logger anymore.
    • Adds escaping before writing to the buffer.

    This is the new benchmarks:

    2022-06-30_17-16