71 Go Distributed-systems Libraries

Kubernetes operator for YTsaurus.

yt-k8s-operator YTsaurus is a distributed storage and processing platform for big data with support for MapReduce model, a distributed file system and

01

A tool for importing secrets from a pre-existing secrets management systems (e.g. Vault, Secrets Manager) into a SealedSecret

kubeseal-convert The missing part of Sealed Secrets. 🔐 Motivation kubeseal-convert aims to reduce the friction of importing secrets from a pre-existi

02

Operational Systems @ UCU course project

Operational Systems @ UCU course project

Clon Clon (an Rclone analogue) Third-term Operational Systems course project Request Feature Table of Contents About The Project Built With Usage Road

03

A distributed architecture with multi-option of transporting data

Microservices with Go The Microservices Architecture contains Broker Service Authentication Service Logger Service Mail Service Listener Service AMQP

04

A high availability distributed filesystem built on FoundationDB and fuse.

High Availability Filesystem An experimental and vapourware distributed filesystem library and fuse filesystem built on FoundationDB. Goals Motivation

05

Tracetest - the best way to develop and test your distributed system with OpenTelemetry

Tracetest - the best way to develop and test your distributed system with OpenTelemetry

Tracetest - the best way to develop and test your distributed system with OpenTelemetry. For QA, Dev, & Ops. Install | Documentation | Twitter | Disco

06

MIT 6.824: Distributed Systems (Spring 2020)

MIT6.824 MIT 6.824: Distributed Systems (Spring 2020) Lab 1 Lab 2 Lab 2A Lab 2B Lab 2C Lab 2D Lab 3 Lab 3A Lab 3B Lab 4 Lab 4A Lab 4B Lab 4 Challenge

07

Scalable distributed log storage for strong consistency, total order, and high availability

Varlog Varlog is a strongly consistent distributed log storage. It enables many distributed systems to leverage total-ordered logs to support transact

09

Lightning powered distributed virtual private network with Bitcoin and Lightning integration.

Indra Lightning powered distributed virtual private network with Bitcoin and Lightning integration. About White Paper The ubiquitous use of encryption

10

❄ A distributed generator to create unique IDs with ease; inspired by Twitter's Snowflake

❄ A distributed generator to create unique IDs with ease; inspired by Twitter's Snowflake

❄ Spaceflake ❄ A distributed generator to create unique IDs with ease; inspired by Twitter's Snowflake. What is a Snowflake? Apart from being a crysta

12

A Fork of gocelery - Go Client/Server for Celery Distributed Task Queue

A Fork of gocelery - Go Client/Server for Celery Distributed Task Queue

gocelery.v3 A Fork of gocelery Go Client/Server for Celery Distributed Task Queue Why? Having been involved in several projects migrating servers from

13

Lightning powered distributed virtual private network with Bitcoin and Lightning integration.

Indranet Lightning powered distributed virtual private network with Bitcoin and Lightning integration. About White Paper The ubiquitous use of encrypt

14

A cascading stream forwarding unit for scalable, distributed voice and video conferencing over Matrix

Waterfall Why Waterfall is a cascaded, decentralised SFU. The intention is to be a implementation of Matrix's MSC3401: Native Group VoIP signalling. T

15

Distributed cache with client-side consistent hashing, distributed leader-elections, and dynamic node discovery. Supports both REST and gRPC interfaces secured with mTLS.

Distributed cache with client-side consistent hashing, distributed leader-elections, and dynamic node discovery. Supports both REST and gRPC interfaces secured with mTLS.

minicache Distributed cache implemented in Go. Like Redis but simpler. Features include: Client-side consistent hashing to support fault-tolerance by

16

A simple, customisable distributed job/worker in Go

A simple, customisable distributed job/worker in Go

Tasqueue is a simple, lightweight distributed job/worker implementation in Go Installation go get -u github.com/kalbhor/tasqueue Concepts Server Optio

17

The First Redis Cache Library To Ensure Eventual Consistency And Strong Consistency With DB.

The First Redis Cache Library To Ensure Eventual Consistency And Strong Consistency With DB.

English | 简体中文 RocksCache The first Redis cache library to ensure eventual consistency and strong consistency with DB. Features Eventual Consistency:

18

A db proxy for distributed transaction, read write splitting and sharding! Support any language! It can be deployed as a sidecar in a pod.

A db proxy for distributed transaction, read write splitting and sharding! Support any language! It can be deployed as a sidecar in a pod.

DBPack DBPack is a database proxy pack aims to provide a distributed transaction solution of high performance and has no invasion into business logic.

19

Post-money economy simulator in the form of a free fictional game based on gift economy that tests the hypothesis of having a market system without any exchange be it barters or money.

Post-money economy simulator in the form of a free fictional game based on gift economy that tests the hypothesis of having a market system without any exchange be it barters or money.

Post-money economy simulator in the form of a free fictional game based on gift economy that tests the hypothesis of having a market system without any exchange be it barters or money.

20

A flexible multi-layer Go caching library to deal with in-memory and shared cache by adopting Cache-Aside pattern.

A flexible multi-layer Go caching library to deal with in-memory and shared cache by adopting Cache-Aside pattern.

go-cache Photo by Ashley McNamara, via ashleymcnamara/gophers (CC BY-NC-SA 4.0) A flexible multi-layered caching library interacts with private (in-me

21

A facade that simplifies interaction with multiple types of file systems.

filestore The filestore package provides a facade that encapsulates the common operations of a readable/writable file system. This could be the actual

22

BANANO's Distributed Proof of Work System (BoomPoW v2)

This is a distributed proof of work system for the BANANO and NANO cryptocurrencies. What is It? Banano transactions require a "proof of work" in orde

23

Cancellable, Efficient and Reliable Distributed Task Queue in Go

Reliable, Efficient and Cancellable Distributed Task Queue in Go NQ ( Nats Queue ) is Go package for queuing and processing jobs in background with wo

24

A distributed C++ compiler: like distcc, but faster

A distributed C++ compiler: like distcc, but faster

nocc — a distributed C++ compiler nocc propagates a compiler invocation to a remote machine: nocc g++ 1.cpp calls g++ remotely, not locally. nocc spee

25

🤹 MIT 6.824 Distributed Systems

🤹 MIT 6.824 Distributed Systems

MIT 6.824 Distributed Systems, Spring 2022 Raft election log Labs Warning All code only for reference. DO NOT copy if you are required to write your o

26

A Go library for microservice registry and executing RPC (Remote Procedure Call) over Redis.

Inventa for Go A Go library for microservice registry and executing RPC (Remote Procedure Call) over Redis. WHY THIS PROJECT? Service discovery, regis

27

Redislocker is a set of distributed locks with high availability and high concurrency based on redis + lua.

Redislocker English | 简体中文 Redislocker is a set of distributed locks with high availability and high concurrency based on redis + lua. Redislocker cur

28

Add, remove, and manage different versions of web-distributed software binaries. No elevated permissions required!

Add, remove, and manage different versions of web-distributed software binaries. No elevated permissions required!

A cross-platform package manager for the web! Add, remove, and manage different versions of web-distributed software binaries. No elevated permissions

29

🐱Kitten is a distributed file system optimized for small file storage, inspired by Facebook's Haystack.

kitten 🐱 简体中文 About kitten Kitten is a distributed file system optimized for small files storage,inspired by Facebook‘s Haystack。 Features High throu

30

A package manager for unix-like systems written in Golang

⚠️ No longer maintained ⚠️ This is no longer maintained, please consider using RedsonBr140/gpkg Fpkg A package manager for Unix-like Operating systems

31

Candado - Global lock service that provides synchronization for distributed systems.

Candado Candado, provides a global lock service for distributed systems. Use Cases Accessing a shared resource Imagine a scenario where you have a sha

32

Final project for Engineering Distributed Infrastructure course.

Final project for Engineering Distributed Infrastructure course.

DS Course Project: Alerting Platform Final project of Distributed Systems course done together with @olearczuk and @brzezinskimarcin. Project overview

33

Distributed network capture (aka: ditributed tcpdump)

Distributed tcpdump ddump is a small utility to run multiple network captures on multiple targets and live-merging the captured streams. This is usefu

34

DisGo: a sample distributed and event-driven application

Designing Microservices Platforms with NATS This the code repository is based on Designing Microservices Platforms with NATS, published by Packt. What

35

Implemented a distributed system for managing tokens

Implemented a distributed system for managing tokens

Implemented a distributed system for managing tokens. Existence and Replication schemes of all tokens are static and known apriori to all the server and client nodes in your system. All tokens are initially created and replicated by the token management launcher whi ch also initializes all the serves. Further, Clients can issue RPC calls to the server to execute read or write methods on tokens.

36

Building highly available, scalable, resilient distributed applications using Go

Microservices with Go Building highly available, scalable, resilient distributed applications using Go Applications Front End service, that just displ

37

A distributed ID generator implemented in Go using the Twitter snowflake algorithm.

snowflake Install Set up your Go environment and run go get github.com/GrokkingSystemDesign/snowflake Usage package main import ( "fmt" "github.co

38

A ligthweight container for distributed security policy evaluation.

A ligthweight container for distributed security policy evaluation.

Rönd Rönd is a ligthweight container that allows you to distribute security policy enforcing throughout your application. Rönd is based on OpenPolicy

39

A distributed lock idea implemented by redis and golang.It is more secure than a simple redis distributed lock.

Redis-ReplicaLock A distributed lock idea implemented by redis and golang.It is more secure than a simple redis distributed lock. It is still in the e

40

Golang distributed Slowloris attack 🦥

Golang distributed Slowloris attack 🦥

slowloris - Golang distributed Slowloris attack How it works Read the article 🦷 How to protect from it TBD Installation Run go install github.com/its

41

Distributed key value in Go

dkvg Description Run Multiple clients REPL (local development) dkvg Description distributed key value in Go Run Multiple clients

42

Distributed key-value store Database

GolemDB Golem is a key-value store based on the consensus algorithm raft. Idears Logs single logs are stored as blocks links are made by first create

43

Distributed commit logging service (Kafka clone)

Distributed Logging A commit logging distributed services with Go. Why build this? What's new here? Simple reason is that distributed systems are cool

44

Distributed unique ID generator library for in-application embedded use

unqid Warning: This is a pre-alpha software. Things are going to change drastically while we develop towards v1. Unqid is a distributed unique ID gene

45

🔒 The golang implemetation of Redis distributed locks

gredlock The golang redlock algorithm implementation. Installation go get -u github.com/vvanglro/[email protected] Quickstart package main import ( "c

46

high-performance non-intrusive distributed transaction solution, inspired by kubernetes, only for golang language.

hptx High-performance non-intrusive distributed transaction solution, inspired by kubernetes, only for golang language. Features High-performance and

47

Raft + Postgres parser + Postgres wire protocol + Boltdb + ? = Distributed PostgreSQL

WaterbugDB Terminal 1: $ go build $ ./waterbugdb --node-id node1 --raft-port 2222 --http-port 8222 --pg-port 6000 Terminal 2: $ go build $ ./waterbugd

48

A light weight distributed key-value database system

flowdb Setup # build go build -o flowdb-server cmd/server/main.go go build -o flowdb-client cmd/client/main.go # run cluster ./flowdb-server --server

49

sovler for formal systems, inspired by GEB

GEB Formal System Solver This code was inspired by a challenge presented in Douglas Hofstadter's book, Godel, Escher, Bach. This challenge comes in th

50

Experiment voting system that aim to realize some typical fault tolerance implementation in a scalable, distributed environment

Fault Tolerant Computing Term Project Setup Dev Environment node 17.7.2 go 1.18 protoc 3.6.1 OSX # Install Golang compiler & toolchain # Mac - instal

51

Robust nim client with failure detection, failover support, and distributed tracing

Robust nim client with failure detection, failover support, and distributed tracing

Robust Nim Client Robust nim client with failure detection, failover support, and distributed tracing. Built as an assignment for UBC's CPSC 416 (Dist

52

A globally, 64 bits, thread-safe identifier for Go.

Seq A globally, 64 bits, thread-safe identifier for Go. It can generate 4,194,303 numbers per second. Introduction ┌--------┬--------┬--------┬-------

53

A toy distributed hash table based on a B-Tree.

A Toy Distributed Hash Table Based on a B-Tree. This is really NOT for production right now, and it doesn't ever intend to be. There are multiple serv

54

A simple search-and-recommend distributed database for education purpose and just entertainment

SarDB(Search-and-Recommend Database) This DB is a simple search-and-recommend oriented distributed database (maybe) for education purpose or just ente

55

BAG:EL - A distributed graph processing system to run PageRank and shortest path calculations, made as a group project for UBC's CPSC 416 - Distributed Systems.

BAG:EL Best Algorithms for Graphs: Easy Learning A distributed graph processor based on the Pregel API. Possible operations are: finding the shortest

56

Simple batch interpreter for the most operating systems. (Linux, Windows, macOS)

Simple batch interpreter for the most operating systems. (Linux, Windows, macOS)

BatchPlus Simple batch interpreter for the most operating systems. (Linux, Windows, macOS) Features Virtual windows CMD shell windows batch file inter

57

Coding exercises from the book "Distributed Services with Go" by Travis Jeffrey, March 2021, ISBN: 9781680507607

dist-services-with-go Coding exercises and notes from the book "Distributed Services with Go" by Travis Jeffrey, March 2021, ISBN: 9781680507607 Book

58

TinySQL is a course designed to teach you how to implement a distributed relational database in Go

TinySQL TinySQL is a course designed to teach you how to implement a distributed relational database in Go. TinySQL is also the name of the simplifed

59

Simple HTTP/HTTPS proxy - designed to be distributed as a self-contained binary that can be dropped in anywhere and run.

Simple Proxy This is a simple HTTP/HTTPS proxy - designed to be distributed as a self-contained binary that can be dropped in anywhere and run. Code b

60

Resilient, Distributed Transactions

RDTS (Resilient Distributed Transactions) Introduction The goal of RDTS is to provide a reliable pattern for guaranteeing ACD transactions (atomic, co

61

Monorepo implementing Boba, a compute layer for Ethereum

Monorepo implementing Boba, a compute layer for Ethereum

The Boba Monorepo TL;DR Documentation Community and DAO Directory Structure Contributing Dependencies Spinning up the stack Helpful commands Running u

62

Go double cache system in memory(Ristretto) and distributed(redis)

Go Supercache A double cache system (memory and redis) to fast and distribuited cache. Install $ go get github.com/alboompro/fastcache Usage package m

63

A distributed rate limit tool: Based on redis, multi-type implementation, easy to use and high performance.

ratelimit (Under development) This package provides a Golang implementation of the Redis-based distributed rate limit algorithm, including Sliding Win

64

Automated penetration and auxiliary systems, providing XSS, XXE, DNS log, SSRF, RCE, web netcat and other Servers

Automated penetration and auxiliary systems, providing XSS, XXE, DNS log, SSRF, RCE, web netcat and other Servers

Automated penetration and auxiliary systems, providing XSS, XXE, DNS log, SSRF, RCE, web netcat and other Servers

65

fleet ties together systemd and etcd into a distributed init system

Deprecation warning fleet is no longer developed or maintained by CoreOS. After February 1, 2018, a fleet container image will continue to be availabl

66

Enable requests served by caddy for distributed tracing via The OpenTracing Project.

caddy-opentracing Enable requests served by caddy for distributed tracing via The OpenTracing Project. Dependencies The Go OpenTracing Library Jaeger,

67

Framework for building distributed services with Web Assembly

Framework for building distributed services with Web Assembly

Tarmac Framework for building distributed services with Web Assembly Tarmac is a unique framework designed for the next generation of distributed syst

68

Golimit is Uber ringpop based distributed and decentralized rate limiter

Golimit is Uber ringpop based distributed and decentralized rate limiter

Golimit A Distributed Rate limiter Golimit is Uber ringpop based distributed and decentralized rate limiter. It is horizontally scalable and is based

69

golang实现的分布式唯一ID生成器distributed id generator,有全局趋势递增、严防时钟漂移、高可用、高性能等特点

golang实现的分布式唯一ID生成器distributed id generator,有全局趋势递增、严防时钟漂移、高可用、高性能等特点

ekko-idgenerator是什么 顾名思义,ekko是一个分布式唯一ID生成器,参考了snowFlake思想,但是并不局限于其设计。 名称由来 英雄联盟的时间刺客ekko 特点 易用,最大限度保证系统的易用性,支持Get与MultiGet; 高并发,单机每秒100w个唯一ID生成; 高可用,理

70

Distributed lock manager. Warning: very hard to use it properly. Not because it's broken, but because distributed systems are hard. If in doubt, do not use this.

What Dlock is a distributed lock manager [1]. It is designed after flock utility but for multiple machines. When client disconnects, all his locks are

71
1