Technotes

Technotes for future me

Filesystem

See Also:

Misc

  • detox: Tool for recursive cleanup of file names.

    detox -v -r <directory>
    
  • Fast File Deletion:

    perl -e 'for(<*>){((stat)[9]<(unlink))}'
    
  • POSIX ACLs:

    getfacl <file>           # List ACLs for file 
    setfacl -m user:joe:rwx dir # Modify ACL
    ls -ld <file>            # Check for active ACL (indicates a "+")
    
  • uNetBootin: Create bootable media for any distribution. Most useful with USB sticks.

  • rsync - –delete doesn’t work: It happens when you call rsync without a trailing slash in the source path like this:

    rsync -az -e ssh --delete /data server:/data
    

    It just won’t delete anything. It will when running it like this:

    rsync -az -e ssh --delete /data/ server:/data
    
  • List physical disk serial number

      # As root
      hdparm -I /dev/sda | grep Serial
      lshw -class disk
      smartctl -i /dev/sda
      lsblk --nodeps -o name,serial
    
      # As user
      /sbin/udevadm info --query=property --name=sda |grep SERIAL
    
  • Debug drive events

      udevadm monitor            # Track any changes
    
      udevadm test /devices/pci0000:00/<id>
    
  • inotify - Detect file access

      apt-get install inotify-tools
    
      inotifywait -m -r /var/log           # Show all activity in all subdirs
      inotifywait -m -e create -r /data    # Show all file/dir creations
      inotifywait -m -e isdir -r /data     # Show all subdir accesses (e.g. before mkdir)
    
      inotifywait -m /var/log/auth.log     # Show all access to single file
    
  • losetup

    losetup /dev/loop0 <disk image>      # Create
    losetup -e /dev/loop0 <disk image>   # Create with encryption
    losetup /dev/loop0          # Get info on loop0
    losetup -d /dev/loop0           # Detach
    losetup -a              # List used loop devices
    
  • dm-crypt with/without LUKS

Automated Sync

Synching without a distributed filesystem

  • lsyncd: inotify based rsync daemon

    lsyncd -rsync /home remotehost.org::share/ 
    

    Or in foreground

    lsyncd -nodaemon -log all -rsyncssh <local dir> <remote host> <remote dir>
    

Distributed Filesystems

NameVendorDistinguishing Features
HDFSHadoopFUSE, NFS, HFTP, S3
CephRedhatNFS, POSIX, S3, Swift, Fuse, Docker support
GlusterFSRedhatNFS, POSIX, S3, Docker support, also as “Redhat Storage Server”
OCFS2OraclePOSIX, often used with DRBD Block replication
LustreOraclePOSIX, abandoned, used in HPC
MooseFSCore TechnologyPOSIX, NFS drop-in

Cloud/SaaS

NameVendorDistinguishing Features
Elastifile ECFSGoogle Cloud, AWS, Azure
NoobaRedhatEnd-user easy cluster on any cloud storage
Azure Netapp FilesAzure, Google Cloud

SSD Erase

hdparm --user-master u --security-set-pass ctmagazin /dev/sdX
hdparm --user-master u --security-erase ctmagazin /dev/sdX
Last updated on 31 Jan 2021
Published on 25 Dec 2019
Edit on GitHub