Network Block Device in Linux

Network Block Devices

This tutorial is based on my talk in ILUGC meet on Feb 13 2010.

What is NBD?
NBD is a software which enables one computer to provide another computer a direct low-level access to a block device. In other terms it helps share a block device over network.

Advantages
In  some environment the performance of sharing over network is better than NFS.( Sometimes even 10x it is claimed). This is commonly seen in thinclient.
It can export any type of filesystem
Best used as swap over network

Installation
NBD has two components. nbd-server and nbd-client.

On debian/ubuntu installation is like this

apt-get install nbd-server
apt-get install nbd-client

nbd-server is required on the server sytem.
nbd-client is required on the client system

Usage
On debian/ubuntu you can start relevant services like this

—————————————————-
# /etc/init.d/nbd-server start

** (process:4393): WARNING **: Could not parse config file: Could not open config file.
** Message: Nothing to do! Bye!
nbd-server.
—————————————————-
Do not bother about the Warning. This will be issued if no configuration file present.  You can give all required paramaters in command line.
Start the client
————————————–
#/etc/init.d/nbd-client start
Starting NBD client process: Connecting…Activating…
nbd-client.

————————————–
This generally creates block devices /dev/nbd0 .. nbd10 for later usage.

I will explore nbd usage under following scenario

1.Server and client will be save.

2.Use localhost (127.0.0.1) for ip address.

Exercise 1
Create an ext2 filesystem in a file. Mount this as block device. I will use /mnt/exports/nbd-exports as file which will have an ext2 file sytem. You can use other partition also

————————————–
# mkdir /mnt/exports
————————————–
use dd to create a file of 10mb

————————————–
# dd if=/dev/zero of=/mnt/exports/nbd-export bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB) copied, 1.47414 s, 69.5 MB/s
————————————–

Now create a filesystem in a file

————————————–
# mke2fs /mnt/exports/nbd-export
mke2fs 1.41.3 (12-Oct-2008)
/mnt/exports/nbd-export is not a block special device.
Proceed anyway? (y,n)

Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
25064 inodes, 100000 blocks
5000 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67371008
13 block groups
8192 blocks per group, 8192 fragments per group
1928 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729

Writing inode tables:  0/13 1/13      2/13      3/13      4/13      5/13      6/13      7/13      8/13      9/13     10/13     11/13     12/13     done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 26 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

————————————–

create nbd server
Syntax of nbd-server is
nbd-server ip:portno file -l [permitted host]

So let me start the server

————————————–
# nbd-server 5000 /mnt/exports/nbd-export
** (process:4441): WARNING **: Could not parse config file: Could not open config file.

————————————–
Here 5000 denotes portnumber. /mnt/exports/nbd-export is the file/block device
we want to share with others.

Now I will use nbd-client to mount. For this first we have to start nbd-client process to attach to the server. Here it is

————————————–
# nbd-client 127.0.0.1 5000 /dev/nbd0
Negotiation: ..size = 100000KB
bs=1024, sz=100000
————————————–
Here    :

127.0.0.1 denotes server’s IP address
5000 denotes server port number
/dev/nbd0 is the device on the client.
Once the above command is executed, nbd-client will talk to server
and negotiate appropriate communication mechanism. This we can see
as the output above

Now that the device is created, I can mount it like other devices.

————————————–
# mount /dev/nbd0 /mnt/test
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda2             19220528   9270540   8973640  51% /
tmpfs                   513888         0    513888   0% /lib/init/rw
udev                     10240        84     10156   1% /dev
tmpfs                   513888         0    513888   0% /dev/shm
/dev/sda3             56765020  51267864   2613492  96% /home
/dev/scd0              1231890   1231890         0 100% /media/cdrom0
/dev/nbd0                96828      1550     90278   2% /mnt/test
————————————–
Now we can use the mounted device as regular part of our system

————————————–
# ls -l /mnt/test
total 12
drwx—— 2 root root 12288 2010-02-14 09:50 lost+found

# cp /tmp/testfile /mnt/test

————————————–

If you want to disconnect following steps should be followed.

First Unmount the device
————————————–
# umount /mnt/test
————————————–
Then disconnect with -d option

————————————–
# nbd-client -d /dev/nbd0
Disconnecting: que, disconnect, sock, done
Kernel call returned: Broken pipeClosing: que, sock, done
————————————–

Exercise 2

Here I want to export an iso image to client as an ISO file sytem.
I have debian dump in /home/lenny_stable/

The sequence of commands are
Start the server
————————————–
# nbd-server 8000 /home/lenny_stable/debian-500-i386-DVD-1.iso
** (process:4456): WARNING **: Could not parse config file: Could not open config file.
————————————–

Here 8000 is the port number.

Now connect the client

————————————–
# nbd-client 127.0.0.1 8000 /dev/nbd1
Negotiation: ..size = 4588206KB
bs=1024, sz=4588206
————————————–

————————————–
# mount /dev/nbd1 /mnt/test1
mount: block device /dev/nbd1 is write-protected, mounting read-only
————————————–
Now we can use this like regular device
————————————–
# ls -l /mnt/test1
total 1068
-r–r–r– 1 root root     27 2009-02-14 22:24 autorun.inf
dr-xr-xr-x 2 root root   2048 2009-02-14 22:24 css
lr-xr-xr-x 1 root root      1 2009-02-14 22:24 debian -> .
-r–r–r– 1 root root    984 2009-02-07 00:05 dedication.txt
dr-xr-xr-x 3 root root   2048 2009-02-14 22:24 dists
dr-xr-xr-x 6 root root   6144 2009-02-14 22:24 doc
-r–r–r– 1 root root  56513 2009-01-23 22:13 g2ldr
-r–r–r– 1 root root   8192 2009-01-23 22:13 g2ldr.mbr
dr-xr-xr-x 2 root root   2048 2009-02-14 22:24 install
dr-xr-xr-x 3 root root   2048 2009-02-14 22:24 install.386
dr-xr-xr-x 5 root root   4096 2009-02-14 22:24 isolinux
-r–r–r– 1 root root 509091 2009-02-14 22:32 md5sum.txt
dr-xr-xr-x 2 root root   4096 2009-02-14 22:24 pics
dr-xr-xr-x 4 root root   2048 2009-02-14 22:25 pool
-r–r–r– 1 root root   9714 2009-02-14 22:32 README.html
-r–r–r– 1 root root 119979 2009-02-14 01:22 README.mirrors.html
-r–r–r– 1 root root  60194 2009-02-14 01:22 README.mirrors.txt
-r–r–r– 1 root root    398 2009-02-14 22:24 README.source
-r–r–r– 1 root root   6150 2009-02-14 22:32 README.txt
-r–r–r– 1 root root 292416 2009-01-23 22:13 setup.exe
dr-xr-xr-x 2 root root   2048 2009-02-14 22:24 tools
-r–r–r– 1 root root    237 2009-02-14 22:24 win32-loader.ini
# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda2             19220528   9270548   8973632  51% /
tmpfs                   513888         0    513888   0% /lib/init/rw
udev                     10240        84     10156   1% /dev
tmpfs                   513888         0    513888   0% /dev/shm
/dev/sda3             56765020  51267868   2613488  96% /home
/dev/scd0              1231890   1231890         0 100% /media/cdrom0
/dev/nbd1              4588206   4588206         0 100% /mnt/test1

————————————–

To disconnect do umount and nbd-client -d

Exercise  3

Directly mounting a block device. I have /dev/sda1 as an ntfs partition.  I want to mount it as  /mnt/tmp. Here it is

————————————–

# nbd-server 9003 /dev/sda1

————————————–

Now connect the nbd-client

# nbd-client 127.0.0.1 9003  /dev/nbd5

Negotiation: ..size = 17583111KB
bs=1024, sz=17583111

————————————–

Now mount it as /mnt/tmp

————————————–

# mount  /dev/nbd5 /mnt/tmp

#ls -l  /mnt/tmp/*.sys

-rw——- 1 root root 1064751104 2009-05-08 09:36 /mnt/tmp/hiberfil.sys
-rw——- 1 root root 1598029824 2009-05-08 09:36 /mnt/tmp/pagefile.sys

————————————–

You can disconnect as usual.

Disadvantages
Allowing write access to more than one client simultaneously may lead to data corruption.
Accessing from server causes deadlock conditions

Links which helped me understand NBD

Network Block Devices: Using Hardware Over a Network

Click to access Network_Block_Devices.pdf