`”We’re Running Out Of Room!”

pace management for a database doesn’t stop with the datafiles as the file system can also present challenges for the DBA. Even though sufficient space may exist inside datafiles the file system itself may be an issue. Depending upon the operating system in use utilities and commands may be available to diagnose such problems. Let’s look at the Linux/U)NIX operating systems and see what they offer.

The most basic command to report disk space is df, which displays various information about one or more mounted file systems. It’s use is simple:

$ df

Typical output is shown below:

Filesystem 1K-blocks Used Available Use% Mounted on
devtmpfs 64938036 0 64938036 0% /dev
tmpfs 64979944 0 64979944 0% /dev/shm
tmpfs 64979944 3457632 61522312 6% /run
tmpfs 64979944 0 64979944 0% /sys/fs/cgroup
/dev/nvme0n1p2 209702892 82315092 127387800 40% /
/dev/nvme4n1 20961280 2529568 18431712 13% /var
/dev/nvme9n1 10475520 1480616 8994904 15% /tmp
/dev/nvme5n1 20961280 449020 20512260 3% /home
/dev/nvme1n1 10475520 106980 10368540 2% /var/tmp
/dev/nvme6n1 31447040 21430336 10016704 69% /opt
/dev/nvme2n1 10475520 4429264 6046256 43% /var/log
/dev/nvme7n1 10475520 8232828 2242692 79% /var/log/audit
/dev/nvme3n1 16644784028 6943909888 8889933796 44% /backup
/dev/nvme8n1 17111375772 14625941024 1626424908 90% /u02
tmpfs 12995988 0 12995988 0% /run/user/1000

The header provides the basic descriptions of the data to follow. In this example the values are reported in kilobytes, but this may not be true for other installations. Three useful switches are available:

-k
-m
-h

which display the output size values in kilobytes, megabytes or in human-readable form. Let’s give each one a test, first df -k:

$ df -k
Filesystem 1K-blocks Used Available Use% Mounted on
devtmpfs 64938036 0 64938036 0% /dev
tmpfs 64979944 0 64979944 0% /dev/shm
tmpfs 64979944 3465824 61514120 6% /run
tmpfs 64979944 0 64979944 0% /sys/fs/cgroup
/dev/nvme0n1p2 209702892 82316100 127386792 40% /
/dev/nvme4n1 20961280 2529548 18431732 13% /var
/dev/nvme9n1 10475520 1480564 8994956 15% /tmp
/dev/nvme5n1 20961280 449020 20512260 3% /home
/dev/nvme1n1 10475520 106980 10368540 2% /var/tmp
/dev/nvme6n1 31447040 21409276 10037764 69% /opt
/dev/nvme2n1 10475520 4427520 6048000 43% /var/log
/dev/nvme7n1 10475520 8232828 2242692 79% /var/log/audit
/dev/nvme3n1 16644784028 6943909888 8889933796 44% /backup
/dev/nvme8n1 17111375772 14625941024 1626424908 90% /u02
tmpfs 12995988 0 12995988 0% /run/user/1000

Now let’s run df -m:

$ df -m
Filesystem 1M-blocks Used Available Use% Mounted on
devtmpfs 63417 0 63417 0% /dev
tmpfs 63457 0 63457 0% /dev/shm
tmpfs 63457 3385 60073 6% /run
tmpfs 63457 0 63457 0% /sys/fs/cgroup
/dev/nvme0n1p2 204788 80388 124401 40% /
/dev/nvme4n1 20470 2472 17999 13% /var
/dev/nvme9n1 10230 1446 8785 15% /tmp
/dev/nvme5n1 20470 439 20032 3% /home
/dev/nvme1n1 10230 105 10126 2% /var/tmp
/dev/nvme6n1 30710 20908 9803 69% /opt
/dev/nvme2n1 10230 4316 5915 43% /var/log
/dev/nvme7n1 10230 8040 2191 79% /var/log/audit
/dev/nvme3n1 16254672 6781162 8681576 44% /backup
/dev/nvme8n1 16710328 14283146 1588306 90% /u02
tmpfs 12692 0 12692 0% /run/user/1000

Notice the values are reported in megabytes which may make them easier to read and less cumbersome to deal with. looking now at df -h we see:

$ df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 62G 0 62G 0% /dev
tmpfs 62G 0 62G 0% /dev/shm
tmpfs 62G 3.4G 59G 6% /run
tmpfs 62G 0 62G 0% /sys/fs/cgroup
/dev/nvme0n1p2 200G 79G 122G 40% /
/dev/nvme4n1 20G 2.5G 18G 13% /var
/dev/nvme9n1 10G 1.5G 8.6G 15% /tmp
/dev/nvme5n1 20G 439M 20G 3% /home
/dev/nvme1n1 10G 105M 9.9G 2% /var/tmp
/dev/nvme6n1 30G 21G 9.6G 69% /opt
/dev/nvme2n1 10G 4.3G 5.8G 43% /var/log
/dev/nvme7n1 10G 7.9G 2.2G 79% /var/log/audit
/dev/nvme3n1 16T 6.5T 8.3T 44% /backup
/dev/nvme8n1 16T 14T 1.6T 90% /u02
tmpfs 13G 0 13G 0% /run/user/1000

The precision of the values is reduced but the units abbreviation is displayed allowing the DBA to see, at a quick glance, the storage numbers.

The Use% column is arguably the most important value found in this output as it reveals the total used space on the device. The Avail column displays the free space remaining for the device; the Use% value is computed from the Size and Avail columns. As can be seen the df command is a good starting point for space investigations.

As good as df is it may not provide the depth of information the DBA may need to pinpoint the source of space issues. Enter the du command, a utility that provides information on every object stored in a given directory or on an entire file system. Usually invoked with the -a switch du reports object sizes in blocks along with a grand total at the end of the output. Examining the /dev file system with du – a reveals:

du -a

0 ./vcsa6
0 ./vcs6
0 ./snd/seq
0 ./snd/timer
0 ./snd
0 ./vhost-vsock
0 ./vhost-net
0 ./vhci
0 ./vfio/vfio
0 ./vfio
0 ./uinput
0 ./ppp
0 ./initctl
0 ./hugepages
0 ./mqueue
0 ./disk/by-label/root
0 ./disk/by-label
0 ./disk/by-partuuid/6264d520-3fb9-423f-8ab8-7a0a8e3d3562
0 ./disk/by-partuuid/fac7f1fb-3e8d-4137-a512-961de09a5549
0 ./disk/by-partuuid
0 ./disk/by-uuid/3276edf0-0d19-46db-826e-b5cf9cfd7b91
0 ./disk/by-uuid/209b92d1-3b0e-4ae9-b097-6f1a28febc31
0 ./disk/by-uuid/076f218c-9306-4532-b805-33d413d60a51
0 ./disk/by-uuid/39447692-bf1a-41f6-b579-41452c622067
0 ./disk/by-uuid/20e84628-8c68-465a-9edd-ef116d18efda
0 ./disk/by-uuid/68c15f1a-2be1-43bb-b94c-29dd8757bc24
0 ./disk/by-uuid/3e0db4d4-1971-456a-b6d6-530797fe4d13
0 ./disk/by-uuid/c0a1cf76-5b5e-4e6a-b7f1-d928452ecb14
0 ./disk/by-uuid/dbd9bce3-60fc-4bee-a6bf-9df2d0111acb
0 ./disk/by-uuid/9143e9e3-9fbf-4e77-bf82-466129c9ebfa
0 ./disk/by-uuid/aba1f778-fd32-4bf5-9919-18c8cebd559f
0 ./disk/by-uuid
0 ./disk/by-path/pci-0000:00:1e.0-nvme-1
0 ./disk/by-path/pci-0000:00:04.0-nvme-1-part2
0 ./disk/by-path/pci-0000:00:04.0-nvme-1-part1
0 ./disk/by-path/pci-0000:00:17.0-nvme-1
0 ./disk/by-path/pci-0000:00:04.0-nvme-1
0 ./disk/by-path/pci-0000:00:16.0-nvme-1
0 ./disk/by-path/pci-0000:00:1d.0-nvme-1
0 ./disk/by-path/pci-0000:00:19.0-nvme-1
0 ./disk/by-path/pci-0000:00:1c.0-nvme-1
0 ./disk/by-path/pci-0000:00:1f.0-nvme-1
0 ./disk/by-path/pci-0000:00:18.0-nvme-1
0 ./disk/by-path/pci-0000:00:1b.0-nvme-1
0 ./disk/by-path/pci-0000:00:1a.0-nvme-1
0 ./disk/by-path
0 ./disk/by-id/wwn-nvme.1d0f-766f6c3065313635613235353936306635613736-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001-part1
0 ./disk/by-id/wwn-nvme.1d0f-766f6c3065313635613235353936306635613736-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001-part2
0 ./disk/by-id/nvme-nvme.1d0f-766f6c3035333436363932613663316666313466-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001
0 ./disk/by-id/nvme-Amazon_Elastic_Block_Store_vol05346692a6c1ff14f
0 ./disk/by-id/nvme-nvme.1d0f-766f6c3065313635613235353936306635613736-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001-part2
0 ./disk/by-id/nvme-Amazon_Elastic_Block_Store_vol0e165a255960f5a76-part2
0 ./disk/by-id/nvme-nvme.1d0f-766f6c3065313635613235353936306635613736-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001-part1
0 ./disk/by-id/nvme-Amazon_Elastic_Block_Store_vol0e165a255960f5a76-part1
0 ./disk/by-id/nvme-nvme.1d0f-766f6c3032356238323634323231323365663038-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001
0 ./disk/by-id/nvme-Amazon_Elastic_Block_Store_vol025b826422123ef08
0 ./disk/by-id/nvme-nvme.1d0f-766f6c3065313635613235353936306635613736-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001
0 ./disk/by-id/nvme-Amazon_Elastic_Block_Store_vol0e165a255960f5a76
0 ./disk/by-id/nvme-nvme.1d0f-766f6c3039353230326531333131653764333435-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001
0 ./disk/by-id/nvme-Amazon_Elastic_Block_Store_vol095202e1311e7d345
0 ./disk/by-id/nvme-nvme.1d0f-766f6c3035343837386435333238326633383634-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001
0 ./disk/by-id/nvme-Amazon_Elastic_Block_Store_vol054878d53282f3864
0 ./disk/by-id/nvme-nvme.1d0f-766f6c3065613230613733613639646139623836-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001
0 ./disk/by-id/nvme-Amazon_Elastic_Block_Store_vol0ea20a73a69da9b86
0 ./disk/by-id/nvme-nvme.1d0f-766f6c3032373332373465373565386138366538-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001
0 ./disk/by-id/nvme-Amazon_Elastic_Block_Store_vol0273274e75e8a86e8
0 ./disk/by-id/nvme-nvme.1d0f-766f6c3062656531663461323431353434386531-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001
0 ./disk/by-id/nvme-Amazon_Elastic_Block_Store_vol0bee1f4a2415448e1
0 ./disk/by-id/nvme-nvme.1d0f-766f6c3037356566623137326166363433386235-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001
0 ./disk/by-id/nvme-Amazon_Elastic_Block_Store_vol075efb172af6438b5
0 ./disk/by-id/nvme-nvme.1d0f-766f6c3030303637653037363063613866303064-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001
0 ./disk/by-id/nvme-Amazon_Elastic_Block_Store_vol00067e0760ca8f00d
0 ./disk/by-id/nvme-nvme.1d0f-766f6c3038626562633137353662303736626533-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001
0 ./disk/by-id/nvme-Amazon_Elastic_Block_Store_vol08bebc1756b076be3
0 ./disk/by-id
0 ./disk
0 ./block/259:8
0 ./block/259:2
0 ./block/259:1
0 ./block/259:6
0 ./block/259:0
0 ./block/259:3
0 ./block/259:11
0 ./block/259:9
0 ./block/259:5
0 ./block/259:4
0 ./block/259:7
0 ./block/259:12
0 ./block/259:10
0 ./block
0 ./ng4n1
0 ./ng8n1
0 ./ng9n1
0 ./ng3n1
0 ./ng6n1
0 ./ng5n1
0 ./nvme6n1
0 ./nvme9n1
0 ./nvme5n1
0 ./nvme8n1
0 ./nvme3n1
0 ./nvme4n1
0 ./ng7n1
0 ./ng2n1
0 ./nvme2n1
0 ./nvme7n1
0 ./ng10n1
0 ./ng1n1
0 ./nvme10n1
0 ./nvme1n1
0 ./ng0n1
0 ./nvme0n1p2
0 ./nvme0n1p1
0 ./nvme0n1
0 ./nvme10
0 ./nvme9
0 ./nvme8
0 ./nvme7
0 ./nvme6
0 ./nvme5
0 ./nvme4
0 ./nvme3
0 ./nvme2
0 ./nvme1
0 ./nvme0
0 ./rtc
0 ./char/7:134
0 ./char/7:6
0 ./char/13:68
0 ./char/242:10
0 ./char/242:4
0 ./char/242:8
0 ./char/242:2
0 ./char/242:9
0 ./char/242:5
0 ./char/242:6
0 ./char/242:7
0 ./char/242:3
0 ./char/242:0
0 ./char/242:1
0 ./char/243:9
0 ./char/243:8
0 ./char/243:7
0 ./char/243:10
0 ./char/243:5
0 ./char/243:6
0 ./char/243:4
0 ./char/243:3
0 ./char/243:2
0 ./char/243:0
0 ./char/243:1
0 ./char/7:129
0 ./char/7:128
0 ./char/7:1
0 ./char/7:0
0 ./char/246:0
0 ./char/4:9
0 ./char/4:8
0 ./char/4:7
0 ./char/4:63
0 ./char/4:62
0 ./char/4:61
0 ./char/4:60
0 ./char/4:6
0 ./char/4:59
0 ./char/4:58
0 ./char/4:57
0 ./char/4:56
0 ./char/4:55
0 ./char/4:54
0 ./char/4:53
0 ./char/4:52
0 ./char/4:51
0 ./char/4:50
0 ./char/4:5
0 ./char/4:49
0 ./char/4:48
0 ./char/4:47
0 ./char/4:46
0 ./char/4:45
0 ./char/4:44
0 ./char/4:43
0 ./char/4:42
0 ./char/4:41
0 ./char/4:40
0 ./char/4:4
0 ./char/4:39
0 ./char/4:38
0 ./char/4:37
0 ./char/4:36
0 ./char/4:35
0 ./char/4:34
0 ./char/4:33
0 ./char/4:32
0 ./char/4:31
0 ./char/4:30
0 ./char/4:3
0 ./char/4:29
0 ./char/4:28
0 ./char/4:27
0 ./char/4:26
0 ./char/4:25
0 ./char/4:24
0 ./char/4:23
0 ./char/4:22
0 ./char/4:21
0 ./char/4:20
0 ./char/4:2
0 ./char/4:19
0 ./char/4:18
0 ./char/4:17
0 ./char/4:16
0 ./char/4:15
0 ./char/4:14
0 ./char/4:13
0 ./char/4:12
0 ./char/4:11
0 ./char/4:10
0 ./char/4:1
0 ./char/4:0
0 ./char/5:0
0 ./char/5:2
0 ./char/5:1
0 ./char/162:0
0 ./char/202:9
0 ./char/202:8
0 ./char/202:7
0 ./char/202:6
0 ./char/202:5
0 ./char/202:4
0 ./char/202:3
0 ./char/202:2
0 ./char/202:15
0 ./char/202:14
0 ./char/202:13
0 ./char/202:12
0 ./char/202:11
0 ./char/202:10
0 ./char/202:1
0 ./char/202:0
0 ./char/10:63
0 ./char/10:231
0 ./char/10:144
0 ./char/10:227
0 ./char/10:183
0 ./char/10:228
0 ./char/10:62
0 ./char/10:235
0 ./char/1:5
0 ./char/1:9
0 ./char/1:8
0 ./char/1:4
0 ./char/1:3
0 ./char/1:11
0 ./char/1:1
0 ./char/1:7
0 ./char/13:63
0 ./char/203:9
0 ./char/203:8
0 ./char/203:7
0 ./char/203:6
0 ./char/203:5
0 ./char/203:4
0 ./char/203:3
0 ./char/203:2
0 ./char/203:15
0 ./char/203:14
0 ./char/203:13
0 ./char/203:12
0 ./char/203:11
0 ./char/203:10
0 ./char/203:1
0 ./char/203:0
0 ./char/4:64
0 ./char/4:67
0 ./char/4:66
0 ./char/4:65
0 ./char/13:66
0 ./char/251:0
0 ./char/13:65
0 ./char/13:64
0 ./char/10:236
0 ./char/13:32
0 ./char/13:67
0 ./char
0 ./uhid
0 ./mapper/control
0 ./mapper
0 ./net/tun
0 ./net
0 ./loop-control
0 ./fuse
0 ./log
0 ./stderr
0 ./stdout
0 ./stdin
0 ./fd
0 ./core
0 ./pts/0
0 ./pts/ptmx
0 ./pts
0 ./shm/orahpatch_CH33
0 ./shm
0 ./cpu_dma_latency
0 ./mcelog
0 ./rtc0
0 ./input/event4
0 ./input/by-path/platform-pcspkr-event-spkr
0 ./input/by-path/platform-i8042-serio-0-event-kbd
0 ./input/by-path/platform-i8042-serio-1-event-mouse
0 ./input/by-path/platform-i8042-serio-1-mouse
0 ./input/by-path
0 ./input/event3
0 ./input/mouse0
0 ./input/event2
0 ./input/event1
0 ./input/event0
0 ./input/mice
0 ./input
0 ./usbmon0
0 ./nvram
0 ./hpet
0 ./raw/rawctl
0 ./raw
0 ./ttyS3
0 ./ttyS2
0 ./ttyS1
0 ./ttyS0
0 ./ptmx
0 ./autofs
0 ./snapshot
0 ./cpu/15/cpuid
0 ./cpu/15/msr
0 ./cpu/15
0 ./cpu/14/cpuid
0 ./cpu/14/msr
0 ./cpu/14
0 ./cpu/13/cpuid
0 ./cpu/13/msr
0 ./cpu/13
0 ./cpu/12/cpuid
0 ./cpu/12/msr
0 ./cpu/12
0 ./cpu/11/cpuid
0 ./cpu/11/msr
0 ./cpu/11
0 ./cpu/10/cpuid
0 ./cpu/10/msr
0 ./cpu/10
0 ./cpu/9/cpuid
0 ./cpu/9/msr
0 ./cpu/9
0 ./cpu/8/cpuid
0 ./cpu/8/msr
0 ./cpu/8
0 ./cpu/7/cpuid
0 ./cpu/7/msr
0 ./cpu/7
0 ./cpu/6/cpuid
0 ./cpu/6/msr
0 ./cpu/6
0 ./cpu/5/cpuid
0 ./cpu/5/msr
0 ./cpu/5
0 ./cpu/4/cpuid
0 ./cpu/4/msr
0 ./cpu/4
0 ./cpu/3/cpuid
0 ./cpu/3/msr
0 ./cpu/3
0 ./cpu/2/cpuid
0 ./cpu/2/msr
0 ./cpu/2
0 ./cpu/1/cpuid
0 ./cpu/1/msr
0 ./cpu/1
0 ./cpu/0/cpuid
0 ./cpu/0/msr
0 ./cpu/0
0 ./cpu
0 ./hwrng
0 ./tty63
0 ./tty62
0 ./tty61
0 ./tty60
0 ./tty59
0 ./tty58
0 ./tty57
0 ./tty56
0 ./tty55
0 ./tty54
0 ./tty53
0 ./tty52
0 ./tty51
0 ./tty50
0 ./tty49
0 ./tty48
0 ./tty47
0 ./tty46
0 ./tty45
0 ./tty44
0 ./tty43
0 ./tty42
0 ./tty41
0 ./tty40
0 ./tty39
0 ./tty38
0 ./tty37
0 ./tty36
0 ./tty35
0 ./tty34
0 ./tty33
0 ./tty32
0 ./tty31
0 ./tty30
0 ./tty29
0 ./tty28
0 ./tty27
0 ./tty26
0 ./tty25
0 ./tty24
0 ./tty23
0 ./tty22
0 ./tty21
0 ./tty20
0 ./tty19
0 ./tty18
0 ./tty17
0 ./tty16
0 ./tty15
0 ./tty14
0 ./tty13
0 ./tty12
0 ./tty11
0 ./tty10
0 ./tty9
0 ./tty8
0 ./tty7
0 ./tty6
0 ./tty5
0 ./tty4
0 ./tty3
0 ./tty2
0 ./tty1
0 ./vcsa1
0 ./vcs1
0 ./vcsa
0 ./vcs
0 ./tty0
0 ./console
0 ./tty
0 ./kmsg
0 ./urandom
0 ./random
0 ./full
0 ./zero
0 ./port
0 ./null
0 ./mem
0 ./vga_arbiter
0 .

The /dev/file system contains entries for all configured devices for the server; since device configurations consume no space the block counts are all 0. For other file systems will have non-zero values. Since du -a reports on all objects in the specified location it scn show which objects are consuming the largest amounts of space, better directing the DBA to areas of concern.

As good as du and df are neither one provides a date/time stamp showing which files/objects have been most recently modified. To get that information a common command, ls, is used. Command switches of interest for such an investigation are:

l long listing, including file size and modification date
t order the results by time, most recently modified on top
r reverse the time ordering
R perform a recursive report

Going back to the /dev file system executing an ls -ltrR reports:

$ ls -ltrR
.:
total 0
drwxrwxrwt. 2 root root 40 Dec 27 21:12 mqueue
drwxr-xr-x. 18 root root 360 Dec 27 21:12 cpu
drwxr-xr-x. 2 root root 60 Dec 27 21:12 raw
drwxr-xr-x. 2 root root 0 Dec 27 21:12 pts
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 stdout -> /proc/self/fd/1
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 stdin -> /proc/self/fd/0
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 stderr -> /proc/self/fd/2
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 fd -> /proc/self/fd
lrwxrwxrwx. 1 root root 11 Dec 27 21:12 core -> /proc/kcore
lrwxrwxrwx. 1 root root 28 Dec 27 21:12 log -> /run/systemd/journal/dev-log
crw——-. 1 root root 10, 239 Dec 27 21:12 uhid
drwxr-xr-x. 2 root root 60 Dec 27 21:12 net
drwxr-xr-x. 2 root root 60 Dec 27 21:12 mapper
drwxr-xr-x. 7 root root 140 Dec 27 21:12 disk
drwxr-xr-x. 2 root root 300 Dec 27 21:12 block
drwxr-xr-x. 2 root root 0 Dec 27 21:12 hugepages
lrwxrwxrwx. 1 root root 12 Dec 27 21:12 initctl -> /run/initctl
crw——-. 1 root root 10, 223 Dec 27 21:12 uinput
crw——-. 1 root root 108, 0 Dec 27 21:12 ppp
crw——-. 1 root root 10, 241 Dec 27 21:12 vhost-vsock
crw——-. 1 root root 10, 238 Dec 27 21:12 vhost-net
crw——-. 1 root root 10, 137 Dec 27 21:12 vhci
drwxr-xr-x. 2 root root 60 Dec 27 21:12 vfio
drwxr-xr-x. 2 root root 80 Dec 27 21:12 snd
crw-rw—-. 1 root disk 10, 237 Dec 27 21:12 loop-control
crw-rw-rw-. 1 root root 10, 229 Dec 27 21:12 fuse
crw——-. 1 root root 243, 0 Dec 27 21:12 nvme0
crw——-. 1 root root 243, 1 Dec 27 21:12 nvme1
crw——-. 1 root root 243, 4 Dec 27 21:12 nvme4
crw——-. 1 root root 243, 3 Dec 27 21:12 nvme3
crw——-. 1 root root 243, 2 Dec 27 21:12 nvme2
crw——-. 1 root root 242, 3 Dec 27 21:12 ng2n1
crw——-. 1 root root 242, 1 Dec 27 21:12 ng1n1
crw——-. 1 root root 242, 0 Dec 27 21:12 ng0n1
crw——-. 1 root root 243, 7 Dec 27 21:12 nvme7
crw——-. 1 root root 243, 5 Dec 27 21:12 nvme5
crw——-. 1 root root 242, 7 Dec 27 21:12 ng3n1
crw——-. 1 root root 243, 6 Dec 27 21:12 nvme6
brw-rw—-. 1 root disk 259, 0 Dec 27 21:12 nvme0n1
crw——-. 1 root root 242, 4 Dec 27 21:12 ng7n1
crw——-. 1 root root 242, 5 Dec 27 21:12 ng5n1
crw——-. 1 root root 242, 9 Dec 27 21:12 ng4n1
crw——-. 1 root root 243, 9 Dec 27 21:12 nvme9
crw——-. 1 root root 243, 8 Dec 27 21:12 nvme8
crw——-. 1 root root 242, 6 Dec 27 21:12 ng6n1
crw——-. 1 root root 242, 8 Dec 27 21:12 ng9n1
crw——-. 1 root root 242, 10 Dec 27 21:12 ng8n1
crw——-. 1 root root 243, 10 Dec 27 21:12 nvme10
brw-rw—-. 1 root disk 259, 2 Dec 27 21:12 nvme0n1p2
crw——-. 1 root root 242, 2 Dec 27 21:12 ng10n1
crw-rw—-. 1 root dialout 4, 67 Dec 27 21:12 ttyS3
crw-rw—-. 1 root dialout 4, 66 Dec 27 21:12 ttyS2
crw-rw—-. 1 root dialout 4, 65 Dec 27 21:12 ttyS1
crw——-. 1 root root 251, 0 Dec 27 21:12 rtc0
lrwxrwxrwx. 1 root root 4 Dec 27 21:12 rtc -> rtc0
brw-rw—-. 1 root disk 259, 1 Dec 27 21:12 nvme0n1p1
brw-rw—-. 1 root disk 259, 12 Dec 27 21:12 nvme6n1
brw-rw—-. 1 root disk 259, 10 Dec 27 21:12 nvme5n1
drwxr-xr-x. 3 root root 200 Dec 27 21:12 input
brw-rw—-. 1 root disk 259, 7 Dec 27 21:12 nvme3n1
brw-rw—-. 1 root disk 259, 11 Dec 27 21:12 nvme8n1
brw-rw—-. 1 root disk 259, 8 Dec 27 21:12 nvme9n1
brw-rw—-. 1 root disk 259, 6 Dec 27 21:12 nvme2n1
brw-rw—-. 1 root disk 259, 5 Dec 27 21:12 nvme7n1
brw-rw—-. 1 root disk 259, 9 Dec 27 21:12 nvme4n1
brw-rw—-. 1 root disk 259, 3 Dec 27 21:12 nvme1n1
brw-rw—-. 1 root disk 259, 4 Dec 27 21:12 nvme10n1
crw-rw-rw-. 1 root root 1, 9 Dec 27 21:12 urandom
crw-rw-rw-. 1 root root 1, 8 Dec 27 21:12 random
crw-r—–. 1 root kmem 1, 4 Dec 27 21:12 port
crw-rw-rw-. 1 root root 1, 3 Dec 27 21:12 null
crw-r—–. 1 root kmem 1, 1 Dec 27 21:12 mem
crw-r–r–. 1 root root 1, 11 Dec 27 21:12 kmsg
crw-rw-rw-. 1 root root 1, 7 Dec 27 21:12 full
crw-rw-rw-. 1 root root 1, 5 Dec 27 21:12 zero
crw——-. 1 root root 10, 144 Dec 27 21:12 nvram
crw——-. 1 root root 10, 227 Dec 27 21:12 mcelog
crw——-. 1 root root 10, 183 Dec 27 21:12 hwrng
crw——-. 1 root root 10, 228 Dec 27 21:12 hpet
crw——-. 1 root root 10, 62 Dec 27 21:12 cpu_dma_latency
crw-r–r–. 1 root root 10, 235 Dec 27 21:12 autofs
crw——-. 1 root root 10, 63 Dec 27 21:12 vga_arbiter
crw——-. 1 root root 10, 231 Dec 27 21:12 snapshot
crw-rw-rw-. 1 root tty 5, 0 Dec 27 21:12 tty
crw——-. 1 root root 5, 1 Dec 27 21:12 console
crw–w—-. 1 root tty 4, 16 Dec 27 21:12 tty16
crw–w—-. 1 root tty 4, 13 Dec 27 21:12 tty13
crw–w—-. 1 root tty 4, 12 Dec 27 21:12 tty12
crw–w—-. 1 root tty 4, 11 Dec 27 21:12 tty11
crw–w—-. 1 root tty 4, 10 Dec 27 21:12 tty10
crw–w—-. 1 root tty 4, 1 Dec 27 21:12 tty1
crw–w—-. 1 root tty 4, 0 Dec 27 21:12 tty0
crw–w—-. 1 root tty 4, 23 Dec 27 21:12 tty23
crw–w—-. 1 root tty 4, 22 Dec 27 21:12 tty22
crw–w—-. 1 root tty 4, 21 Dec 27 21:12 tty21
crw–w—-. 1 root tty 4, 20 Dec 27 21:12 tty20
crw–w—-. 1 root tty 4, 2 Dec 27 21:12 tty2
crw–w—-. 1 root tty 4, 19 Dec 27 21:12 tty19
crw–w—-. 1 root tty 4, 18 Dec 27 21:12 tty18
crw–w—-. 1 root tty 4, 17 Dec 27 21:12 tty17
crw–w—-. 1 root tty 4, 15 Dec 27 21:12 tty15
crw–w—-. 1 root tty 4, 14 Dec 27 21:12 tty14
crw–w—-. 1 root tty 4, 3 Dec 27 21:12 tty3
crw–w—-. 1 root tty 4, 29 Dec 27 21:12 tty29
crw–w—-. 1 root tty 4, 28 Dec 27 21:12 tty28
crw–w—-. 1 root tty 4, 27 Dec 27 21:12 tty27
crw–w—-. 1 root tty 4, 26 Dec 27 21:12 tty26
crw–w—-. 1 root tty 4, 25 Dec 27 21:12 tty25
crw–w—-. 1 root tty 4, 24 Dec 27 21:12 tty24
crw–w—-. 1 root tty 4, 37 Dec 27 21:12 tty37
crw–w—-. 1 root tty 4, 36 Dec 27 21:12 tty36
crw–w—-. 1 root tty 4, 35 Dec 27 21:12 tty35
crw–w—-. 1 root tty 4, 34 Dec 27 21:12 tty34
crw–w—-. 1 root tty 4, 33 Dec 27 21:12 tty33
crw–w—-. 1 root tty 4, 32 Dec 27 21:12 tty32
crw–w—-. 1 root tty 4, 31 Dec 27 21:12 tty31
crw–w—-. 1 root tty 4, 30 Dec 27 21:12 tty30
crw–w—-. 1 root tty 4, 44 Dec 27 21:12 tty44
crw–w—-. 1 root tty 4, 43 Dec 27 21:12 tty43
crw–w—-. 1 root tty 4, 42 Dec 27 21:12 tty42
crw–w—-. 1 root tty 4, 41 Dec 27 21:12 tty41
crw–w—-. 1 root tty 4, 40 Dec 27 21:12 tty40
crw–w—-. 1 root tty 4, 4 Dec 27 21:12 tty4
crw–w—-. 1 root tty 4, 39 Dec 27 21:12 tty39
crw–w—-. 1 root tty 4, 38 Dec 27 21:12 tty38
crw–w—-. 1 root tty 4, 52 Dec 27 21:12 tty52
crw–w—-. 1 root tty 4, 51 Dec 27 21:12 tty51
crw–w—-. 1 root tty 4, 50 Dec 27 21:12 tty50
crw–w—-. 1 root tty 4, 5 Dec 27 21:12 tty5
crw–w—-. 1 root tty 4, 49 Dec 27 21:12 tty49
crw–w—-. 1 root tty 4, 48 Dec 27 21:12 tty48
crw–w—-. 1 root tty 4, 47 Dec 27 21:12 tty47
crw–w—-. 1 root tty 4, 46 Dec 27 21:12 tty46
crw–w—-. 1 root tty 4, 45 Dec 27 21:12 tty45
crw–w—-. 1 root tty 4, 59 Dec 27 21:12 tty59
crw–w—-. 1 root tty 4, 58 Dec 27 21:12 tty58
crw–w—-. 1 root tty 4, 57 Dec 27 21:12 tty57
crw–w—-. 1 root tty 4, 56 Dec 27 21:12 tty56
crw–w—-. 1 root tty 4, 55 Dec 27 21:12 tty55
crw–w—-. 1 root tty 4, 54 Dec 27 21:12 tty54
crw–w—-. 1 root tty 4, 53 Dec 27 21:12 tty53
crw–w—-. 1 root tty 4, 9 Dec 27 21:12 tty9
crw–w—-. 1 root tty 4, 8 Dec 27 21:12 tty8
crw–w—-. 1 root tty 4, 7 Dec 27 21:12 tty7
crw–w—-. 1 root tty 4, 63 Dec 27 21:12 tty63
crw–w—-. 1 root tty 4, 62 Dec 27 21:12 tty62
crw–w—-. 1 root tty 4, 61 Dec 27 21:12 tty61
crw–w—-. 1 root tty 4, 60 Dec 27 21:12 tty60
crw–w—-. 1 root tty 4, 6 Dec 27 21:12 tty6
crw-rw—-. 1 root tty 7, 129 Dec 27 21:12 vcsa1
crw-rw—-. 1 root tty 7, 128 Dec 27 21:12 vcsa
crw-rw—-. 1 root tty 7, 1 Dec 27 21:12 vcs1
crw-rw—-. 1 root tty 7, 0 Dec 27 21:12 vcs
crw——-. 1 root root 246, 0 Dec 27 21:12 usbmon0
crw-rw—-. 1 root tty 7, 134 Dec 27 21:12 vcsa6
crw-rw—-. 1 root tty 7, 6 Dec 27 21:12 vcs6
drwxr-xr-x. 2 root root 3200 Dec 27 21:12 char
crw–w—-. 1 root tty 4, 64 Dec 27 21:12 ttyS0
drwxrwxrwt. 3 root root 60 Jan 3 14:45 shm
crw-rw-rw-. 1 root tty 5, 2 Jan 3 16:01 ptmx

./mqueue:
total 0

./cpu:
total 0
drwxr-xr-x. 2 root root 80 Dec 27 21:12 1
drwxr-xr-x. 2 root root 80 Dec 27 21:12 0
drwxr-xr-x. 2 root root 80 Dec 27 21:12 9
drwxr-xr-x. 2 root root 80 Dec 27 21:12 8
drwxr-xr-x. 2 root root 80 Dec 27 21:12 7
drwxr-xr-x. 2 root root 80 Dec 27 21:12 6
drwxr-xr-x. 2 root root 80 Dec 27 21:12 5
drwxr-xr-x. 2 root root 80 Dec 27 21:12 4
drwxr-xr-x. 2 root root 80 Dec 27 21:12 3
drwxr-xr-x. 2 root root 80 Dec 27 21:12 2
drwxr-xr-x. 2 root root 80 Dec 27 21:12 15
drwxr-xr-x. 2 root root 80 Dec 27 21:12 14
drwxr-xr-x. 2 root root 80 Dec 27 21:12 13
drwxr-xr-x. 2 root root 80 Dec 27 21:12 12
drwxr-xr-x. 2 root root 80 Dec 27 21:12 11
drwxr-xr-x. 2 root root 80 Dec 27 21:12 10

./cpu/1:
total 0
crw——-. 1 root root 203, 1 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 1 Dec 27 21:12 msr

./cpu/0:
total 0
crw——-. 1 root root 203, 0 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 0 Dec 27 21:12 msr

./cpu/9:
total 0
crw——-. 1 root root 203, 9 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 9 Dec 27 21:12 msr

./cpu/8:
total 0
crw——-. 1 root root 203, 8 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 8 Dec 27 21:12 msr

./cpu/7:
total 0
crw——-. 1 root root 203, 7 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 7 Dec 27 21:12 msr

./cpu/6:
total 0
crw——-. 1 root root 203, 6 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 6 Dec 27 21:12 msr

./cpu/5:
total 0
crw——-. 1 root root 203, 5 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 5 Dec 27 21:12 msr

./cpu/4:
total 0
crw——-. 1 root root 203, 4 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 4 Dec 27 21:12 msr

./cpu/3:
total 0
crw——-. 1 root root 203, 3 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 3 Dec 27 21:12 msr

./cpu/2:
total 0
crw——-. 1 root root 203, 2 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 2 Dec 27 21:12 msr

./cpu/15:
total 0
crw——-. 1 root root 203, 15 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 15 Dec 27 21:12 msr

./cpu/14:
total 0
crw——-. 1 root root 203, 14 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 14 Dec 27 21:12 msr

./cpu/13:
total 0
crw——-. 1 root root 203, 13 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 13 Dec 27 21:12 msr

./cpu/12:
total 0
crw——-. 1 root root 203, 12 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 12 Dec 27 21:12 msr

./cpu/11:
total 0
crw——-. 1 root root 203, 11 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 11 Dec 27 21:12 msr

./cpu/10:
total 0
crw——-. 1 root root 203, 10 Dec 27 21:12 cpuid
crw——-. 1 root root 202, 10 Dec 27 21:12 msr

./raw:
total 0
crw-rw—-. 1 root disk 162, 0 Dec 27 21:12 rawctl

./pts:
total 0
c———. 1 root root 5, 2 Dec 27 21:12 ptmx
crw–w—-. 1 ec2-user tty 136, 0 Jan 3 16:01 0

./net:
total 0
crw-rw-rw-. 1 root root 10, 200 Dec 27 21:12 tun

./mapper:
total 0
crw——-. 1 root root 10, 236 Dec 27 21:12 control

./disk:
total 0
drwxr-xr-x. 2 root root 80 Dec 27 21:12 by-partuuid
drwxr-xr-x. 2 root root 60 Dec 27 21:12 by-label
drwxr-xr-x. 2 root root 260 Dec 27 21:12 by-uuid
drwxr-xr-x. 2 root root 300 Dec 27 21:12 by-path
drwxr-xr-x. 2 root root 600 Dec 27 21:12 by-id

./disk/by-partuuid:
total 0
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 6264d520-3fb9-423f-8ab8-7a0a8e3d3562 -> ../../nvme0n1p2
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 fac7f1fb-3e8d-4137-a512-961de09a5549 -> ../../nvme0n1p1

./disk/by-label:
total 0
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 root -> ../../nvme0n1p2

./disk/by-uuid:
total 0
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 209b92d1-3b0e-4ae9-b097-6f1a28febc31 -> ../../nvme0n1p2
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 aba1f778-fd32-4bf5-9919-18c8cebd559f -> ../../nvme5n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 9143e9e3-9fbf-4e77-bf82-466129c9ebfa -> ../../nvme6n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 dbd9bce3-60fc-4bee-a6bf-9df2d0111acb -> ../../nvme3n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 20e84628-8c68-465a-9edd-ef116d18efda -> ../../nvme8n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 3276edf0-0d19-46db-826e-b5cf9cfd7b91 -> ../../nvme9n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 076f218c-9306-4532-b805-33d413d60a51 -> ../../nvme2n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 68c15f1a-2be1-43bb-b94c-29dd8757bc24 -> ../../nvme4n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 3e0db4d4-1971-456a-b6d6-530797fe4d13 -> ../../nvme7n1
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 c0a1cf76-5b5e-4e6a-b7f1-d928452ecb14 -> ../../nvme10n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 39447692-bf1a-41f6-b579-41452c622067 -> ../../nvme1n1

./disk/by-path:
total 0
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 pci-0000:00:04.0-nvme-1 -> ../../nvme0n1
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 pci-0000:00:04.0-nvme-1-part2 -> ../../nvme0n1p2
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 pci-0000:00:04.0-nvme-1-part1 -> ../../nvme0n1p1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 pci-0000:00:1b.0-nvme-1 -> ../../nvme6n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 pci-0000:00:1a.0-nvme-1 -> ../../nvme5n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 pci-0000:00:18.0-nvme-1 -> ../../nvme3n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 pci-0000:00:1d.0-nvme-1 -> ../../nvme8n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 pci-0000:00:1e.0-nvme-1 -> ../../nvme9n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 pci-0000:00:17.0-nvme-1 -> ../../nvme2n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 pci-0000:00:16.0-nvme-1 -> ../../nvme1n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 pci-0000:00:1c.0-nvme-1 -> ../../nvme7n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 pci-0000:00:19.0-nvme-1 -> ../../nvme4n1
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 pci-0000:00:1f.0-nvme-1 -> ../../nvme10n1

./disk/by-id:
total 0
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-nvme.1d0f-766f6c3065313635613235353936306635613736-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 -> ../../nvme0n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-Amazon_Elastic_Block_Store_vol0e165a255960f5a76 -> ../../nvme0n1
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 nvme-nvme.1d0f-766f6c3065313635613235353936306635613736-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001-part2 -> ../../nvme0n1p2
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 nvme-Amazon_Elastic_Block_Store_vol0e165a255960f5a76-part2 -> ../../nvme0n1p2
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 wwn-nvme.1d0f-766f6c3065313635613235353936306635613736-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001-part2 -> ../../nvme0n1p2
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 nvme-nvme.1d0f-766f6c3065313635613235353936306635613736-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001-part1 -> ../../nvme0n1p1
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 nvme-Amazon_Elastic_Block_Store_vol0e165a255960f5a76-part1 -> ../../nvme0n1p1
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 wwn-nvme.1d0f-766f6c3065313635613235353936306635613736-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001-part1 -> ../../nvme0n1p1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-nvme.1d0f-766f6c3030303637653037363063613866303064-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 -> ../../nvme6n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-Amazon_Elastic_Block_Store_vol00067e0760ca8f00d -> ../../nvme6n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-nvme.1d0f-766f6c3038626562633137353662303736626533-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 -> ../../nvme5n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-Amazon_Elastic_Block_Store_vol08bebc1756b076be3 -> ../../nvme5n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-nvme.1d0f-766f6c3037356566623137326166363433386235-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 -> ../../nvme3n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-Amazon_Elastic_Block_Store_vol075efb172af6438b5 -> ../../nvme3n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-nvme.1d0f-766f6c3035343837386435333238326633383634-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 -> ../../nvme8n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-Amazon_Elastic_Block_Store_vol054878d53282f3864 -> ../../nvme8n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-nvme.1d0f-766f6c3035333436363932613663316666313466-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 -> ../../nvme9n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-Amazon_Elastic_Block_Store_vol05346692a6c1ff14f -> ../../nvme9n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-nvme.1d0f-766f6c3032356238323634323231323365663038-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 -> ../../nvme2n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-Amazon_Elastic_Block_Store_vol025b826422123ef08 -> ../../nvme2n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-nvme.1d0f-766f6c3065613230613733613639646139623836-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 -> ../../nvme4n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-nvme.1d0f-766f6c3039353230326531333131653764333435-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 -> ../../nvme1n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-nvme.1d0f-766f6c3032373332373465373565386138366538-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 -> ../../nvme7n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-Amazon_Elastic_Block_Store_vol0ea20a73a69da9b86 -> ../../nvme4n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-Amazon_Elastic_Block_Store_vol095202e1311e7d345 -> ../../nvme1n1
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 nvme-Amazon_Elastic_Block_Store_vol0273274e75e8a86e8 -> ../../nvme7n1
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 nvme-nvme.1d0f-766f6c3062656531663461323431353434386531-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 -> ../../nvme10n1
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 nvme-Amazon_Elastic_Block_Store_vol0bee1f4a2415448e1 -> ../../nvme10n1

./block:
total 0
lrwxrwxrwx. 1 root root 10 Dec 27 21:12 259:0 -> ../nvme0n1
lrwxrwxrwx. 1 root root 12 Dec 27 21:12 259:2 -> ../nvme0n1p2
lrwxrwxrwx. 1 root root 12 Dec 27 21:12 259:1 -> ../nvme0n1p1
lrwxrwxrwx. 1 root root 10 Dec 27 21:12 259:12 -> ../nvme6n1
lrwxrwxrwx. 1 root root 10 Dec 27 21:12 259:10 -> ../nvme5n1
lrwxrwxrwx. 1 root root 10 Dec 27 21:12 259:7 -> ../nvme3n1
lrwxrwxrwx. 1 root root 10 Dec 27 21:12 259:11 -> ../nvme8n1
lrwxrwxrwx. 1 root root 10 Dec 27 21:12 259:8 -> ../nvme9n1
lrwxrwxrwx. 1 root root 10 Dec 27 21:12 259:6 -> ../nvme2n1
lrwxrwxrwx. 1 root root 10 Dec 27 21:12 259:9 -> ../nvme4n1
lrwxrwxrwx. 1 root root 10 Dec 27 21:12 259:5 -> ../nvme7n1
lrwxrwxrwx. 1 root root 10 Dec 27 21:12 259:3 -> ../nvme1n1
lrwxrwxrwx. 1 root root 11 Dec 27 21:12 259:4 -> ../nvme10n1

./hugepages:
total 0

./vfio:
total 0
crw——-. 1 root root 10, 196 Dec 27 21:12 vfio

./snd:
total 0
crw-rw—-. 1 root audio 116, 33 Dec 27 21:12 timer
crw-rw—-. 1 root audio 116, 1 Dec 27 21:12 seq

./input:
total 0
crw-rw—-. 1 root input 13, 32 Dec 27 21:12 mouse0
crw-rw—-. 1 root input 13, 63 Dec 27 21:12 mice
crw-rw—-. 1 root input 13, 67 Dec 27 21:12 event3
crw-rw—-. 1 root input 13, 65 Dec 27 21:12 event1
crw-rw—-. 1 root input 13, 64 Dec 27 21:12 event0
crw-rw—-. 1 root input 13, 66 Dec 27 21:12 event2
crw-rw—-. 1 root input 13, 68 Dec 27 21:12 event4
drwxr-xr-x. 2 root root 120 Dec 27 21:12 by-path

./input/by-path:
total 0
lrwxrwxrwx. 1 root root 9 Dec 27 21:12 platform-i8042-serio-1-mouse -> ../mouse0
lrwxrwxrwx. 1 root root 9 Dec 27 21:12 platform-i8042-serio-1-event-mouse -> ../event3
lrwxrwxrwx. 1 root root 9 Dec 27 21:12 platform-i8042-serio-0-event-kbd -> ../event2
lrwxrwxrwx. 1 root root 9 Dec 27 21:12 platform-pcspkr-event-spkr -> ../event4

./char:
total 0
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 243:0 -> ../nvme0
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 243:1 -> ../nvme1
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 243:4 -> ../nvme4
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 243:3 -> ../nvme3
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 243:2 -> ../nvme2
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 242:1 -> ../ng1n1
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 242:0 -> ../ng0n1
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 243:7 -> ../nvme7
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 243:5 -> ../nvme5
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 242:7 -> ../ng3n1
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 242:3 -> ../ng2n1
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 243:6 -> ../nvme6
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 242:9 -> ../ng4n1
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 242:5 -> ../ng5n1
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 242:4 -> ../ng7n1
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 243:9 -> ../nvme9
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 243:8 -> ../nvme8
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 242:6 -> ../ng6n1
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 242:8 -> ../ng9n1
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 242:10 -> ../ng8n1
lrwxrwxrwx. 1 root root 9 Dec 27 21:12 243:10 -> ../nvme10
lrwxrwxrwx. 1 root root 9 Dec 27 21:12 242:2 -> ../ng10n1
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:66 -> ../ttyS2
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:65 -> ../ttyS1
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 251:0 -> ../rtc0
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:64 -> ../ttyS0
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 13:32 -> ../input/mouse0
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:67 -> ../ttyS3
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 203:2 -> ../cpu/2/cpuid
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 203:15 -> ../cpu/15/cpuid
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 203:12 -> ../cpu/12/cpuid
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 203:11 -> ../cpu/11/cpuid
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 203:1 -> ../cpu/1/cpuid
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 203:0 -> ../cpu/0/cpuid
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 203:9 -> ../cpu/9/cpuid
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 203:8 -> ../cpu/8/cpuid
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 203:7 -> ../cpu/7/cpuid
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 203:6 -> ../cpu/6/cpuid
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 203:5 -> ../cpu/5/cpuid
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 203:4 -> ../cpu/4/cpuid
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 203:3 -> ../cpu/3/cpuid
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 203:14 -> ../cpu/14/cpuid
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 203:13 -> ../cpu/13/cpuid
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 203:10 -> ../cpu/10/cpuid
lrwxrwxrwx. 1 root root 10 Dec 27 21:12 1:9 -> ../urandom
lrwxrwxrwx. 1 root root 9 Dec 27 21:12 1:8 -> ../random
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 1:7 -> ../full
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 1:4 -> ../port
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 13:63 -> ../input/mice
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 1:3 -> ../null
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 1:11 -> ../kmsg
lrwxrwxrwx. 1 root root 6 Dec 27 21:12 1:1 -> ../mem
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 1:5 -> ../zero
lrwxrwxrwx. 1 root root 18 Dec 27 21:12 10:62 -> ../cpu_dma_latency
lrwxrwxrwx. 1 root root 17 Dec 27 21:12 10:236 -> ../mapper/control
lrwxrwxrwx. 1 root root 9 Dec 27 21:12 10:235 -> ../autofs
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 10:228 -> ../hpet
lrwxrwxrwx. 1 root root 9 Dec 27 21:12 10:227 -> ../mcelog
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 10:183 -> ../hwrng
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 202:13 -> ../cpu/13/msr
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 202:12 -> ../cpu/12/msr
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 202:11 -> ../cpu/11/msr
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 202:10 -> ../cpu/10/msr
lrwxrwxrwx. 1 root root 12 Dec 27 21:12 202:1 -> ../cpu/1/msr
lrwxrwxrwx. 1 root root 12 Dec 27 21:12 202:0 -> ../cpu/0/msr
lrwxrwxrwx. 1 root root 14 Dec 27 21:12 10:63 -> ../vga_arbiter
lrwxrwxrwx. 1 root root 11 Dec 27 21:12 10:231 -> ../snapshot
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 10:144 -> ../nvram
lrwxrwxrwx. 1 root root 12 Dec 27 21:12 202:6 -> ../cpu/6/msr
lrwxrwxrwx. 1 root root 12 Dec 27 21:12 202:5 -> ../cpu/5/msr
lrwxrwxrwx. 1 root root 12 Dec 27 21:12 202:4 -> ../cpu/4/msr
lrwxrwxrwx. 1 root root 12 Dec 27 21:12 202:3 -> ../cpu/3/msr
lrwxrwxrwx. 1 root root 12 Dec 27 21:12 202:2 -> ../cpu/2/msr
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 202:15 -> ../cpu/15/msr
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 202:14 -> ../cpu/14/msr
lrwxrwxrwx. 1 root root 12 Dec 27 21:12 202:9 -> ../cpu/9/msr
lrwxrwxrwx. 1 root root 12 Dec 27 21:12 202:8 -> ../cpu/8/msr
lrwxrwxrwx. 1 root root 12 Dec 27 21:12 202:7 -> ../cpu/7/msr
lrwxrwxrwx. 1 root root 13 Dec 27 21:12 162:0 -> ../raw/rawctl
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 5:2 -> ../ptmx
lrwxrwxrwx. 1 root root 10 Dec 27 21:12 5:1 -> ../console
lrwxrwxrwx. 1 root root 6 Dec 27 21:12 5:0 -> ../tty
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:13 -> ../tty13
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:12 -> ../tty12
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:11 -> ../tty11
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:10 -> ../tty10
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 4:1 -> ../tty1
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 4:0 -> ../tty0
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:21 -> ../tty21
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:20 -> ../tty20
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 4:2 -> ../tty2
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:19 -> ../tty19
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:18 -> ../tty18
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:17 -> ../tty17
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:16 -> ../tty16
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:15 -> ../tty15
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:14 -> ../tty14
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 4:3 -> ../tty3
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:29 -> ../tty29
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:28 -> ../tty28
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:27 -> ../tty27
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:26 -> ../tty26
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:25 -> ../tty25
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:24 -> ../tty24
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:23 -> ../tty23
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:22 -> ../tty22
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:36 -> ../tty36
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:35 -> ../tty35
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:34 -> ../tty34
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:33 -> ../tty33
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:32 -> ../tty32
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:31 -> ../tty31
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:30 -> ../tty30
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:43 -> ../tty43
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:42 -> ../tty42
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:41 -> ../tty41
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:40 -> ../tty40
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 4:4 -> ../tty4
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:39 -> ../tty39
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:38 -> ../tty38
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:37 -> ../tty37
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:51 -> ../tty51
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:50 -> ../tty50
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 4:5 -> ../tty5
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:49 -> ../tty49
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:48 -> ../tty48
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:47 -> ../tty47
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:46 -> ../tty46
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:45 -> ../tty45
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:44 -> ../tty44
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:58 -> ../tty58
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:57 -> ../tty57
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:56 -> ../tty56
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:55 -> ../tty55
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:54 -> ../tty54
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:53 -> ../tty53
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:52 -> ../tty52
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 4:9 -> ../tty9
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 4:8 -> ../tty8
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 4:7 -> ../tty7
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:63 -> ../tty63
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:62 -> ../tty62
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:61 -> ../tty61
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:60 -> ../tty60
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 4:6 -> ../tty6
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 4:59 -> ../tty59
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 7:129 -> ../vcsa1
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 7:128 -> ../vcsa
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 7:1 -> ../vcs1
lrwxrwxrwx. 1 root root 6 Dec 27 21:12 7:0 -> ../vcs
lrwxrwxrwx. 1 root root 10 Dec 27 21:12 246:0 -> ../usbmon0
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 13:67 -> ../input/event3
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 13:65 -> ../input/event1
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 13:64 -> ../input/event0
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 13:66 -> ../input/event2
lrwxrwxrwx. 1 root root 15 Dec 27 21:12 13:68 -> ../input/event4
lrwxrwxrwx. 1 root root 7 Dec 27 21:12 7:6 -> ../vcs6
lrwxrwxrwx. 1 root root 8 Dec 27 21:12 7:134 -> ../vcsa6

./shm:
total 0
drwx——. 2 oracle oracle 40 Dec 27 21:13 orahpatch_CH33
ls: cannot open directory ‘./shm/orahpatch_CH33’: Permission denied
[ec2-user@mut01-ora1 dev]$

This can produce a rather extensive listing, however when space issues arise such output can be extremely helpful in identifying a file or set of files causing the problem.

Output from any of these commands can be directed to a file for easier handling. Since /tmp is rarely an issue directing the output from any of these commands to a file on /tmp will keep the output separate from the file system under investigation. Remember to remove the generated file once the investigation has completed and the issue is resolved.

Be mindful of both the configured space and the available space on any device under investigation; the Use% column of df should be carefully considered before making any changes. 90% usage of a 16 TB file system still provides 1.6 TM of space; in such cases it may be better to adjust any thresholds monitoring tools are using to report issues. A 5% threshold on that same 16 TB file system provides a cushion of 800 GB, plenty of space to allow storage to be added before any real emergency would exist.

Space management for database systems doesn’t stop at the tablespace level, so the DBA should be familiar with space management utilities provided by the operating system vendor. Remember, too, that more than one command or function may be useful in investigating space issues; space investigations are progressive, and it may be necessary to dig deeper to find the source and, as a result, a solution. Perseverance can pay big dividends where space is concerned; patience is key to solving many a space issue. Hopefully the tools presented here will be useful to those DBAs working on UNIX/Linux platforms.

Go forth, and conquer.