replicate/nydus-snapshotter

replicate/nydus-snapshotter

Description: A containerd snapshotter with data deduplication and lazy loading in P2P fashion

Language: Go

License: Apache-2.0

Stars: 0

Forks: 0

Open issues: 0

Created: 2023-07-24T17:52:34Z

Pushed: 2023-08-10T13:52:01Z

Default branch: main

Fork: yes

Parent repository: containerd/nydus-snapshotter

Archived: no

README: [[⬇️ Download]](https://github.com/containerd/nydus-snapshotter/releases) [[📖 Website]](https://nydus.dev/) [[☸ Quick Start (Kubernetes)]](https://github.com/containerd/nydus-snapshotter/blob/main/docs/run_nydus_in_kubernetes.md) [[🤓 Quick Start (nerdctl)]](https://github.com/containerd/nerdctl/blob/master/docs/nydus.md) [[❓ FAQs & Troubleshooting]](https://github.com/dragonflyoss/image-service/wiki/FAQ)

Nydus Snapshotter

!CI 

Nydus-snapshotter is a non-core sub-project of containerd.

Nydus snapshotter is an external plugin of containerd for Nydus image service which implements a chunk-based content-addressable filesystem on top of a called RAFS (Registry Acceleration File System) format that improves the current OCI image specification, in terms of container launching speed, image space, and network bandwidth efficiency, as well as data integrity with several runtime backends: FUSE, virtiofs and in-kernel EROFS.

Nydus supports lazy pulling feature since pulling image is one of the time-consuming steps in the container lifecycle. Lazy pulling here means a container can run even the image is partially available and necessary chunks of the image are fetched on-demand. Apart from that, Nydus also supports (e)Stargz and OCI (by using zran) lazy pulling directly WITHOUT any explicit conversion.

For more details about how to build Nydus container image, please refer to nydusify conversion tool and acceld.

Architecture

Architecture Based on FUSE


Architecture Based on Fscache/Erofs


Building

Just invoke make and check out the output executable binary ./bin/containerd-nydus-grpc

make

Integrate Nydus-snapshotter into Containerd

The following document will describe how to manually configure containerd + Nydus snapshotter. If you want to run Nydus snapshotter in Kubernetes cluster, you can try to use helm or run nydus snapshotter as a container. You can refer to [this documentation](./docs/run_nydus_in_kubernetes.md).

Containerd provides a general mechanism to exploit different types of snapshotters. Please ensure your containerd's version is 1.4.0 or above. Add Nydus as a proxy plugin into containerd's configuration file which may be located at /etc/containerd/config.toml.

# The `address` field specifies through which socket snapshotter and containerd communicate.
[proxy_plugins]
[proxy_plugins.nydus]
type = "snapshot"
address = "/run/containerd-nydus/containerd-nydus-grpc.sock"

Restart your containerd service making the change take effect. Assume that your node is systemd based, restart the service as below:

systemctl restart containerd

Get Nydus Binaries

Get nydusd nydus-image and nydusctl binaries from nydus releases page. It's suggested to install the binaries to your system path. nydusd is FUSE userspace daemon and a vhost-user-fs backend. Nydus-snapshotter will fork a nydusd process when necessary.

Configure Nydus

Please follow instructions to [configure nydus](./docs/configure_nydus.md) in order to make it work properly in your environment.

Start Nydus Snapshotter

Nydus-snapshotter is implemented as a proxy plugin (containerd-nydus-grpc) for containerd.

Assume your server is systemd based, install nydus-snapshotter: Note: nydusd and nydus-image should be found from $PATH.

make install
systemctl restart containerd

Or you can start nydus-snapshotter manually.

# `--nydusd` specifies the path to nydusd binary. If `nydusd` and `nydus-image` are installed, `--nydusd` and `--nydus-image`can be omitted.
# Otherwise, provide them in below command line.
# `address` is the domain socket that you configured in containerd configuration file
# `--nydusd-config` is the path to `nydusd` configuration file
# The default nydus-snapshotter work directory is located at `/var/lib/containerd-nydus`

$ sudo ./containerd-nydus-grpc --config /etc/nydus/config.toml --nydusd-config /etc/nydus/nydusd-config.json --log-to-stdout

Validate Nydus-snapshotter Setup

Utilize containerd's ctr CLI command to validate if nydus-snapshotter is set up successfully.

$ ctr -a /run/containerd/containerd.sock plugin ls
TYPE ID PLATFORMS STATUS
io.containerd.snapshotter.v1 nydus - ok

Optimize Nydus Image as per Workload

Nydus usually prefetch image data to local filesystem before a real user on-demand read. It helps to improve the performance and availability. A containerd NRI plugin [container image optimizer](docs/optimize_nydus_image.md) can be used to generate nydus image building suggestions to optimize your nydus image making the nydusd runtime match your workload IO pattern. The optimized nydus image has a better performance.

Quickstart Container with Lazy Pulling

Start Container on single Node

Start container using nerdctl (>=v0.22) which has native nydus support with nydus-snapshotter.

# Start container by `nerdctl`
nerdctl --snapshotter nydus run ghcr.io/dragonflyoss/image-service/nginx:nydus-latest

Start Container in Kubernetes Cluster

Change containerd's CRI configuration:

[plugins."io.containerd.grpc.v1.cri".containerd]
snapshotter =...