• Main
  • Networking
  • Security
  • WLAN
  • Mausezahn
  • Students
  • Other SW
    • Local Anchors:
  • Propaganda
  • Tricks
  • rsync
  • sitecopy
  • convmv
  • Multimedia
  • Virtualization

    • When Linux is your first choice

    I admit that I'm a Linux fan since 1994. So this section is kinda propaganda. Often people ask me why I use Linux as the only operating system on all my computers and how this is practical for my daily work. Here are the reasons:

    Actually there are lots of more advantages (speed, platform portability, old hardware reuse, network and hacker tools) but let's stop the propaganda here.

    I use Linux since April 1994 (SuSE 4.x on those days), then tried out Redhat, DLD, and others until I got stuck with Debian for many years. Recently I switched to Ubuntu because it is the best 'plug and play' Linux I found and has Debian as core. I still use Debian for some servers in parallel.


    Linux (Ubuntu) Tricks

    Below you find my personal notes on several Ubuntu-related issues. For troubleshooting I recommend to consult the Ubuntu Forums and the official User Documentation. If you prefer short explanations you might find the list below useful. Of course most of these notes might be valid for other LINUXes.
    Interesting Shell Commands
    Here are some practical commands which are not so widely known (maybe):
    • whereis <cmd> quickly finds binary, source, manpage for a given command. Example: whereis xterm
    • man -k <keyword> shows a list of all manpages containing the keyword

    Quick TFTP server
    Using the netwox tool (the 'network swiss army knife') you can easily start a TFTP server on demand:
    # netwox 167 --rootdir "/home/herbert/tftp"
    Jed must-have
    My favorite editor is jed, a very small and fast Emacs-clone. Unfortunately with the standard Ubuntu setup it does not show line or column numbers and also the line wrap is too short. You can simply adjust this by creating a .jedrc file with the following content:
    LINENUMBERS = 2; WRAP = 120;
    Postscript Printing
    To print more than one page on a single sheet of paper, use
    $psnup -2 input.ps output.ps
    For M$-Powerpoint presentations:
    $psnup -4 -pa4 -H20cm -W30cm -d present.ps present2.ps

    Recording Internet Radio
    Either use kmplayer or mimms such as:
    mimms mms://stream4.orf.at/oe1-wort
    Setting the clock
    It is highly recommended to configure UTC (Universal Time Coordinated = 'Greenwich') as system time (the hardware clock) while the UNIX operating system (here: Linux) adjusts for the time zone. Using the date command you can specify either local or UTC. Here is an example using the local time:
    date -s 5/22/07 # First set the date (initialized to 00:00:00) date -s 11:42:00 # Or simply 11:42 without seconds
    /etc/fstab
    Ubuntu Edgy (and most other modern distros) no longer uses filepaths for partition entries, instead the UUID is used, which makes the configuration more robust in case additional SCSI disks are inserted. The UUID of a partition can be obtained via blkid (short, disks only) or lshal (more info). For example, on my laptop fstab looks like:
    UUID=6693f472-f7df-4c57-b4db-b74b2e41de22 / ext3 defaults,errors=remount-ro 0 1 UUID=98e42079-522d-4d9c-8c24-87077ef42800 none swap sw 0 0 /dev/scd0 /media/cdrom0 udf,iso9660 user,noauto 0 0 2. field: mountpoint or "none" for swap 3. field: type (ext3, ...) 4. field: mount options 5. field: used by dump, may be omitted 6. field: determines sequence for fsck check during reboot, may be omitted

    convert
    Convert belongs actually to the ImageMagick suite and can be regarded as 'Photoshop on the command line' because you can do nearly anything with images. Here is a nice example I found recently on linuxjournal.com:
    convert -size 800x120 xc:white -font Times-Roman -pointsize 100 \ -fill gray -annotate +20+80 'Linux is cool!' -fill black \ -annotate +23+83 'Linux is cool!' -trim +repage logo.png
    This creates a nice logo on the fly...

    Suspend/Hibernate Issues
    Hint: Examine /etc/default/acpi-support


    RSYNC

    The Backup Problem is one of the oldest problems of man. Basically you want to synchronize some directories between multiple machines, but you want to modify files on any device you like. Sure you do not want to struggle with Windows-based (SMB) file sharing, which is slow (also updates unchanged files) and sometimes the backup fails because of various strange reasons (and you need to repeat the whole process...). Furthermore the backup process should be activated on demand and should run quickly.

    I tried out several solutions, but I found that generally all pretty GUI-based solutions are clumsy, error-prone, complicated and sometimes unstable. Then I tried out rsync. Many people already told me about rsync in positive ways but I always had some reservations. I thought rsync is just another mighty but complicated command line tool.

    But rsync is exactly what you need. I'll give you a practical example how to get a quick setup which I think is appropriate in many situations. Everything I explain works on Ubuntu/Debian machines but should be similar on other Linuxes.

    First (for example) identify a central server, say 'zeus'. Assume that your user/group name is 'homer'. Install the rsync package and then:

    Note: In the configuration example above we created two 'modules' (i. e. pre-defined locations on the rsyncd server).

    Now use rsync. Rsync supports different transport options. The remote-shell transport is used whenever the source or destination path contains a single colon (:) separator after a host specification. I do not use that.

    I rather contact the rsync server directly. This is achieved when the source or destination path contains a double colon (::) separator after a host specification (alternatively, when an rsync:// URL is specified).

    The general syntax is: rsync [OPTION] SRC DEST. Examples:

    marge@laptop:~$ rsync -avz Desktop/dev zeus::desktop # only sync 'dev' marge@laptop:~$ rsync -avz Desktop zeus::desktop # sync whole Desktop marge@laptop:~$ rsync -avzu zeus::desktop/results Desktop # sync from zeus

    If neither the source or destination path specify a remote host, the copy occurs locally, that is you can even use rsync to synchronize local disks for example. I do this very often to backup directories on my external USB-based harddisk. For example:

    marge@laptop:~$ rsync -avz Desktop/documents /media/mydisk



    The most important rsync options are: 

    -a, --archive     archive mode; same as -rlptgoD (no -H)
-v, --verbose     increase verbosity
-u, --update      skip files that are newer on the receiver
--delete          delete files that don’t exist on sender
--force           This option tells rsync to delete a non-empty 
                  directory when it is to be replaced by a non-directory.
-b, --backup      With this option, preexisting destination files are
                  renamed as each file is transferred or deleted.

-C, --cvs-exclude This is a useful shorthand for excluding a broad range
                  of files that you often don’t want to transfer between  
		  systems. It uses the same algorithm that CVS uses to 
		  determine if a file should be ignored.

The exclude list is initialized to:

RCS  SCCS  CVS CVS.adm RCSLOG cvslog.* tags TAGS .make.state
.nse_depinfo *~ #* .#* ,* _$* *$ *.old *.bak *.BAK *.orig
*.rej .del-* *.a *.olb *.o *.obj *.so *.exe *.Z *.elc *.ln core .svn/
    Note that rsync support much more options, I really recommend to read the man page at least once in order to find the most appropriate options for your need. For example I personally most always use the options 'avuz', that is only newer files are updated. To make things simple, I made an alias for this options (alias rsyncsafe='rsync -azuv'), so I only use the pseudocommand 'rsyncsafe' for most of my backups.

    NOTE: The trailing slash in a path means 'the files in that directory' but not the directory itself. So be careful.


    Sitecopy

    While I use rsync to synchronize data between my computers, I use sitecopy to mirror my local website directory to my remote web space. Only differences are transmitted and it supports FTP, WebDAV, or HTTP as transport protocol.

    Here is a short description to get it started:

    1. Create a configuration directory and -file:
      mkdir -m 700 .sitecopy touch .sitecopyrc chmod 600 .sitecopyrc # This config file is NOT in the directory .sitecopy
    2. Edit .sitecopyrc and specify your website(s) such as:
      site perihel server www.perihel.at remote ~/www.perihel.at/ local ~/Desktop/PERIHEL/current/ username XXXXXXX password YYYYYYY safe #protocol webdav symlinks ignore #maintain exclude *~ exclude tmp exclude cgi-bin
    3. Only when the remote site is "empty" let sitecopy learn what is the "empty" state:
      sitecopy --init perihel # make sitecopy think there are NO files \ on the remote server
      You do this only once. If you have already some files on the remote web space then you better do the following:
    4. Sometimes (e. g. when the timestamp of the remote files have been modified) sitecopy must learn about all remote changes using the fetch option:
      sitecopy --fetch perihel # or short option: -f
      ...this only makes a local file list (it does not download anything)
    5. Synchronize the remote web space with the local files - this is the most important and usually only step:
      sitecopy --update perihel # or short option: -u
    6. Other interesting options: -l or -ll produces a list of differences, -o shows the progress.

    convmv

    Converts filenames from one encoding to another. I needed this when I copied mp3-files from Windows to my Linux disks.

    Short description:

    convmv -f latin1 -t utf-8 -r \ --exec "echo #1 should be renamed to #2" path/to/files -f ... from -t ... to -r ... recurse through directories --qfrom , --qto be more quiet about the "from" or "to" of a rename (if it screws up your terminal e. g.). This will in fact do nothing else than replace any non-ASCII character (bytewise) with ? and any control character with * on printout, this does not affect rename operation itself. --list list all available encodings.

    A practical example when converting German Windows filenames (code page 850) to Linux (using UTF-8):

    convmv -f cp850 -t utf-8

    Linux and Multimedia

    One big advantage of Linux is that you do not need to care about DVD regional codes, encryptions, etc. If you install the packages libdvdcss and w32codecs then you can easily access all DVDs (and also rip them). Needless to say: Don't steal movies! Hollywood and their actors are getting poorer and poorer! Have mercy!

    For Ubuntu you could add the Medibuntu repository to install these files.

    Here are some interesting programs and a short description:

    Rippers
    • dvd::rip very good, supports copy on disk first, lots of options, a bit buggy (need newer release)
    • acidrip simple, other codecs such as x264 and lacv (fastest way). Requires mplayer and mencoder.
    • thoggen very simple and convenient - currently only strange (but good!) codecs (ogg/theora) supported but this should change soon
    Codecs
    • xviD Open Source, some DVD Players do not support fast forward and backward.
    • Divx4 means OpenDivX and is also an Open Source Variant but AFAIK DivX 3 and DivX 5 are much better regarding quality and speed. Therefore don't use this codec anymore.
    • x264 reportedly the best codec regarding quality (according professional comparisons). I also noticed that I got the smallest file sizes while keeping same quality. Most DVD players do not support it.
    • lacv this is the default codec for acidrip, not supported by some DVD players, but it is quite fast.
    Transcode is an utility to encode raw video/audio streams. You may want to study the transcode Wiki.

    Virtualization - VMWARE

    [Update Sept09]: I highly recommend VirtualBox 3.0 which I use now for several weeks. It is much better regarding performance, memory footprint, standby/continuation speed, and seemless integration into the Linux desktop.

    If you still need MS-DOS don't struggle with dual-boot configurations anymore. Simply get vmware (server or workstation, NOT the player) and create your own virtual Windows images.

    Fortunately vmware-server is now availably for free and there is even an Ubuntu package for it, for example for 'feisty' add the following line to your repository list (/etc/apt/sources.list):

    deb http://archive.canonical.com/ubuntu feisty-commercial main
    and install vmware-server. First get a valid serial number from www.vmware.com (for free). That's all.

    Alternatively, if you want to compile your own fresh vmware, do the following:

    When vmware-server has been installed successfully then create your image:

    Virtualization - VirtualBox

    Another (more) free solution is VirtualBox which also supports all MS-DOS variants (even Vista) very well. Note that there is an open source and a (more advanced) closed source variant. The main advantage is that you get the feature of a 'shared folder' for free, while, when using VMWARE, only the non-free workstation version supports this.

    Update [Sept09]: I use VirtualBox 3.0 for some times now on my Kubuntu 8.04-laptop and to my opinion it is much better than VMWare solutions. The performance and memory consumption is superior, and the really rocking feature is CTRL-L which integrates (virtual) MSDOS on the Linux desktop in a seamless way.

    ©20070713 by Herbert Haas | imprint