CEG 233: Linux and Windows 

Lab on Installing Linux on a Four GB USB Drive

Table of Contents

  1. What is an OS?
  2. Linux Distributions
  3. Generations of Windows
  4. Overview of the OS-on-USBD
  5. Lab Experiment
  6. Acknowledgements
  7. References

Educational Objectives

The objectives of this lab experiment are to make you :

  1. Feel bolder installing OS other than Windows.
  2. Use Linux more extensively by providing a portable installation.
  3. Understand partitions, file volumes, and boot sectors.
  4. Use systems programs and understand user privileges.

This lab will install a Linux distribution that can bee booted and run entirely from a USB drive.  The Linux system that is so-installed is a full-scale system, suitable for all the CS/CEG courses of WSU.  This lab is done early in the term so that all the students have "their own" Linux that they can run on home PCs without disturbing Windows or other software.

You will understand processes better in a couple of weeks.

Professional Ethics

Unless a PC has disabled booting from a USBD, this lab lets you boot from the USB drive and become the privileged user named root

1. What is an OS?

An OS controls all the hardware of a computer, and every program currently running.  All computer systems, by definition, include an OS.  Several modern day gadgets, such as cell phones, MP3 players, wrist watches, PDAs, video game consoles, and TiVO, are specialized computer systems.  Many appliances, such as microwave ovens, dishwashers and TVs, contain embedded computer systems.

Currently popular OS for personal computers include Windows, Linux, MacOS,  and Solaris. Our discussion in this article is so modern that software known as CP/M (1960s), MS-DOS (1960s) will not qualify as OS.

1.1 Computer Systems

A computer system can be divided into three pieces:

  1. [Hardware]  Motherboard (CPU, RAM, and other chips), various daughter cards (e.g., a graphics card, a wireless network card), keyboard, mouse, screen, hard disks, other IO devices, and power supply.
  2. [Software] Operating system, and applications. 
  3. [Firmware] A basic input/output subsystem (BIOS) preloaded into a ROM

A typical PC fits this classification very well; but, there are many classes of computers that do not.  Some computer systems may not have a key board or a hard disk. 

A program is a sequence of CPU instructions (also called machine code) structured in a very specific way.  Software is a generic synonym for programs.  Applications are a subclass of software.

The CPU can execute its instructions only when they are in RAM.  We have so many programs that it is infeasible to leave them all sitting in RAM.  The OS stores them on HDD and loads them into RAM as needed.

We will explain in the lectures the "Fetch, Decode, and Execute cycle" of a CPU.

CPU can generate traps and faults. 

Devices raise interrupts. Interrupt handling is done by the OS.

The moment a computer system is powered on, the firmware is in control of every piece of the hardware. On a PC, this firmware is called the BIOS (basic input output subsystem).

A computer system must not be shut down "ungracefully".  All the running processes need to be informed of the impending shut down, and wait politely for them to terminate.  This is known as graceful termination of the OS.  Most systems can now detect a power failure, and initiate a shutdown.  However, un-responding processes not withstanding a shutdown cannot be postponed indefinitely, and the OS forcibly terminates.

1.2 OS Definition by Functionality

There are many definitions of OS based on its typical functionality. From the book by Silbershatz and others:  "What are the three main purposes of an operating system? Answer: (i) To provide an environment for a computer user to execute programs on computer hardware in a convenient and efficient manner. (ii) To allocate the separate resources of the computer as needed to solve the problem given. The allocation process should be as fair and efficient as possible. (iii) As a control program it serves two major functions: (a) supervision of the execution of user programs to prevent errors and improper use of the computer, and (b) management of the operation and control of I/O devices."

1.3 OS Definition by Components

A very useful definition of an OS is based on its components.  If we were to spread out all the code that constitutes an OS on a piece of cloth, we can then cut it up into the following components:

OS = VM + FileM + ProcM + Net + IO + UserM + (Boot) + [SysPrg]

Strictly speaking, an OS does not include GUI, nor CLI.  In the world of Linux, the GUI is provided by X11, Gnome and KDE software subsystems, and CLI by bash, csh, zsh or other such shells.  These are not strictly part of the Linux OS.  In the Windows world, the GUI is provided by (plain) Explorer, and the CLI by cmd or PowerShell.  These are not strictly  part of the Windows OS.

Programs like Word, Internet Explorer, Konqueror are applications.  Visual Studio, g++, Active Python and such are (sub-) categorized as development tools.

The above components can also be combined and then re-divided into a kernel + initial ram disk components + drivers + interrupt handlers.

1.4 OS as a Collection of System Calls

The code of system calls is not disjoint from the code of previously mentioned components.  It is made of different pieces from these components.  Linux and Windows have around 350 system calls each. The details of Linux syscalls can be found in the files:

-rw-r--r--  1 root root  1403 2010-01-03 06:57 /usr/include/sys/syscall.h
-rw-r--r--  1 root root 22041 2010-01-28 00:56 /usr/include/asm/unistd_64.h

1.5 OS as a Collection of Files

Linux:
  1. vmlinuz: OS kernel. This is a specially linked file. All of its code is executed in a privileged CPU mode. Its code can be divided up as suggested by the OS = VM + FileM + ProcM + Net + IO + UserM; note that we dropped +(Boot)+[SysPrg] parts. The z at the end in its name indicates that this is a compressed file. The exact location of this file is indicated in the "kernel" line of a GRUB stanza.
  2. initrd.gz: This is a compressed collection of a bunch of system files that the kernel needs as it boots. The exact location of this file is indicated in the "kernel" line of a GRUB stanza.
  3. /lib/modules: This directory contains a number of dynamically loadable "modules", which are specially linked files that can be incorporated into the running kernel.
  4. /sbin: This directory contains a number of system programs that the kernel can invoke and spawn off as separate OS helper processes.
  5. /usr/sbin: This directory contains further system programs. The programs in /sbin are considered essential, whereas the one in /usr/sbin are "less" so.

/bin, /usr/bin, ...: These directories do contain programs, but these are not considered part of the OS. The programs in these directories are considered simply applications.

1.6 OS as a Collection of Processes

Most of the OS stays RAM resident and dormant until (i) a system call is made by a process, (ii) a hardware interrupt causes a handler in the OS to run, or (iii) a software triggered even causes a handler in the OS to run. Consequently, viewing an OS only as a collection of processes is going to be incomplete.

The following is a severely pruned list of processes as generated by the ps command.  The process-id shown (except for 1) may change.  Processes shown within brackets are part of the Kernel.


PIDTTYSTATTIME COMMANDFunctionality
1?Ss0:00/sbin/initThe First Process
2?S<0:00[kthreadd]
3?S<0:00[migration/0] Processes shown within brackets are part of the Kernel.
524?S0:00udevd Dynamic Device Manager service
1053?Ss0:00portmap RPC port map service
1056?Ss0:00dd bs=1 if=/proc/kmsg ... Kernel message logging
1063?Sl0:00rsyslogd 
1088?Ss0:10haldHardware Abstraction Layer
1357?S<0:00[nfsiod]NFS Serivice IO
1403?S0:00hald-addon-cpufreqCPU frequency
1404?S0:00hald-addon-acpi:ACP listening
1479tty4Ss+0:00/sbin/gettyUser login process
1503?Ss0:00acpid 
2159?S0:00dbus-launch 
2738?Ss0:00/usr/sbin/nmbdSamba Name Service
2748?Ss0:00/usr/sbin/smbdSamba File Service

1.7. OS Boot

We will explain in the lectures: Boot sequence.  POST.  Locating bootable media.  Loading the MBR.  Loading the OS boot loader.

1. Powered on.  BIOS->POST. ROM and RAM. CPU fetch-decode-execute cycle.
2. Discover boot device.  Boot sectors. Boot device priority order.
3. Bring in and give control to the boot loader.  Bring in v. "load"
4. Bring in and give control to OS loader.  NTLDR, GRUB, etc.  (http://en.wikipedia.org/wiki/GRUB)
5. OS kernel.  Linux kernel = vmlinuz + initrd
6. For Linux: process init is started.

Process init (in Linux, and its counterpart in Windows) is in control of "normal" operation of a computer.

Cold versus warm boot.  Cold as if power is just applied. Warm skips first few steps.

Ubuntu and other distributions use UUID (Universally Unique Identifier) to indicate the root device.  From the Wiki: In its canonical form, a UUID consists of 32 hexadecimal digits, displayed in 5 groups separated by hyphens, for a total of 36 characters. For example: 550e8400-e29b-41d4-a716-446655440000 .

See Required Reading below.

2. Linux Distributions

There are way too many (about a thousand?) distributions.  No one has a complete list, but the DistroWatch site is quite extensive.  The "major" distributions are (in no particular order): Debian (Ubuntu), RedHat, SuSE, Gentoo, and SlackWare.  Most of the minor distributions are specializations of these.

We are using an ISO customized for OSIS Lab usage. It is about 3.5 GB in size. Its lineage is: osislablinuxlive.iso ⇒ Linux Mint ⇒ Kubuntu Linux ⇒ Debian.

3. Generations of Windows

Windows 3.1, 95, 98 are from one code lineage.  The NT, 2000, XP, Vista and Windows 7 are from a different lineage.

4. Overview of OS-on-USBD

4.1 Installing an OS

There are two ways of installing an OS onto a USB mass storage drive (USBD).  This assumes that your laptop/desktop is recent enough to permit booting from a USBD.

  1. Insert the USBD and start the OS installation.  During the install, select the USBD as the target drive.  Modern OS installation procedures will discover the USBD as if it is a normal HDD, and let you partition, format, and mount the partitions.  This has the advantage of higher speed, but requires a larger USBD.

  2. Boot into a host OS.  It can be Linux or Windows. Transfer the content of an OS LiveCD onto the USBD.  Then, install an OS boot loader on the USBD.    Most live CDs have their larger files compressed, and before they can be used must be uncompressed, either before hand or on the fly as needed.  So, this does not have the same speed as an installation done as in the above.  But, the required size capacity of the USBD is no more than that of the live CD.   Knoppix Live CD compresses about 2 GB worth of programs onto a 700+ MB CD. There are live CDs ranging from 50 MB to 760 MB.  There are live CDs of many OS: Linux, FreeDOS, FreeBSD, OpenBSD, Solaris, and even Windows.  There are now (2010) over a hundred live DVDs.

In this lab, we will be following method 2, using Linux as the host OS.  You will be installing a Linux distribution on your own USBD, which should be at least 1 GB in size.  Be aware that all its content will be destroyed during this lab.  (Note that some (old) USBD are known to have hardware issues when used as bootable devices.)

4.2 Privileged Commands

In this lab, you will be invoking some commands that are privileged because they have a destructive possibility.  You are expected to look up the man pages of sudo, fdisk, mkfs.ext2, mkfs.vfat, and grub.  Be extremely careful in what arguments you supply to these commands; double-check before you press return.

4.3 Overview of the Steps

This section is not a step-by-step procedure.  It explains the behind the scenes reasoning of the steps you will carry out later on. Peculiarly perhaps we are not just interested in creating a bootable USBD; we are more interested in understanding the internal stucture of what directories and files such a USBD has.

  1. [Partitioning the USBD] All mass storage devices can be viewed as a long sequence of sectors/blocks.  These devices can read/write in units of blocks only; they cannot read/write single bytes.  A consecutive section of blocks is a partition.  A large hard disk can be divided into several partitions.  A small disk will be made into single partition.  There are primary, extended and logical partitions.  For simplicitly, we divide the USBD into two primaries.

    We make a VFAT volume on the first partition of the USBD so that the USBD can be readily used in Windows also.

    Linux and hence the partitioning program "think" that the USBD is a HDD, so the use of terms like cylinders etc. should be discounted.

    The partions on a 4 GB USBD should look like the following. Your numbers may not match exactly with these. A block is 1 kB.

    Disk /dev/sdb: 4009 MB, 4009754624 bytes
    126 heads, 22 sectors/track, 2825 cylinders
    Units = cylinders of 2772 * 512 = 1419264 bytes
    Disk identifier: 0x000a8f3b
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sdb1               1         283      392227    c  W95 FAT32 (LBA)
    /dev/sdb2             284        2825     3523212   83  Linux
    
  2. [Making File Volumes on USBD] Before any thing can be stored on a partition, it needs to be initialized.  This is known as making a file volume.  Formatting is a lower level activity, but in the Windows world this often means making a file volume.

  3. [Mounting ] To access the files on a volume, it must be "mounted."  Note that files are persistent on a USBD or HDD, but are not "on-line" until mounted.

  4. [Copy files of LinuxOS] Typical commands that you are familiar with such as ls, mkdir, bash, and the KDE etc. are all packed into one file named filesystem.squashfs.  We can crudely describe this as a zip archive.  More precisely, it is the compressed file-system image containing many files. The Linux kernel consists of two files: vmlinuz and the initial compressed-ram-disk file initrd.gz.

  5. [GRUB Installation]  This step if carried with the slightest of the typos can destroy the hard-disk booting of our lab PCs.  We are using GRUB as our OS loader.

4.4 Use of USBD Linux

The OS-on-USBD you produce at the end of this Lab is will work like any other hard disk installed Linux.  There are several limitations of course: (i) Due to the smaller size of the USBD compared to typical HDD, the range of installed software is limited.  (ii) The speed suffers: (ii-a) USBD are several times slower than HDD, and hence.  (ii-b) Also, there may not be a swap partition.  (ii-c) On the USBD, we use a compressed read-only file system that is uncompressed piece meal as needed. (iii) Because it is read-only, new or updated software cannot be installed.

Once you boot from the USBD, there may be couple of dialogs about keyboard, time zone, etc.  depending on the ISO image we chose.

Some motherboards of desktops/laptops have trouble booting (hangs or reboots) unless acpi is turned off, etc.  So, experiment with the kernel parameters by interacting with GRUB.

As you shutdown, there may be a dialog about other virtual terminals being active. Just ignore this.

Our procedure in creating this USBD mimics a LiveCD. So, there may be a dialog about removing "the CD" and then pressing Enter. Just press Enter. This can be eliminated by appending the token "noprompt" after the token "splash" to the grub menu stanzas.

Our USBD, the way we constructed it, does not save any customization of appearances, etc. that you may make. So, on the next boot from this USBD these are all lost.

After booting from this USBD, you can mount the first (vfat) partition, and save your work there.


5. Lab Experiment

All work is expected, but not required, to be carried out in the Operating Systems and Internet Security (OSIS) Lab, 429 Russ. But, you are welcome to work wherever. Note that use of both Linux and Windows and other software, that may not always be installed in other facilities, may be needed.

It is expected that ypu have understood the previous sections, especially the Overview.

  1. [Partitioning the USBD] Insert your USBD. In the OSIS lab, its device name will be /dev/sdb. If a popup dialog asks you about mounting, do not click on OK; click on Do-Nothing. When doing this experiment elsewhere the name of the USBD may be different.  If you use the wrong name, you will be wiping out what was on the (wrongly) named device. Invoke:
            sudo fdisk /dev/sdb

    1. Delete all existing partitions on your drive. Press d and type the number of the partition to be deleted. Do this for all partitions.

    2. We are going to make two primary partitions.  Make the first partition taking up all the space on the USB leaving 3500+ MB for Linux.

      You type n for new,  p for primary, 1 for partition number, then we specify the needed size.

      The size specification can be confusing mostly because we see "cylinders" and this is a USBD. Accept the default start cylinder of 1. Specify the size using the syntax of "+sizeM", that is, the plus-sign, followed by the needed size as a number, then a big M, all without any spaces in them. E.g., "+390M" (without the double quotes).

    3. Make the second partition to be a 3500+ MB primary partition for Linux. Type n for new, p for primary, 2 for partition number, and accept the defaults.  Make sure that this partition is 3500+ MB in size. In the partition example above, this second partition came out with 3523212 blocks. If not, redo by going back to deleting all the partitions, and choosing a smaller first partition.

    4. Set the type of the first partition to vfat. Type t, 1, c (or 6 etc).

    5. Record a listing of these partitions. Type p for print details of the partitions.

    6. Write to the USBD the internal data structures that the above steps have created: w for write.

  2. [Making File Volumes on USBD] This will take a few seconds.

    1. Create a vfat file volume on the Windows partition. 
          sudo /sbin/mkfs.vfat /dev/sdb1

      Record the output of mkfs.vfat.
    2. Create an ext2 file volume on the Linux partition:
          sudo /sbin/mkfs.ext2 -L casper-rw /dev/sdb2

      Record the output of mkfs.ext2.

  3. [Mounting]
    1. Mount OS image:

          mkdir -p /tmp/P
          sudo mount -o loop,ro /var/ISO/osislablinuxlive.iso /tmp/P
      
      This is the source image of the Linux OS image we are installing.
    2. Mount your USBD:
          mkdir -p /tmp/U
          sudo mount /dev/sdb2 /tmp/U
          df -T
      Record the output of df.

  4. [Copy files of Linux OS]Copy the entire image of Linux OS:
        time sudo cp -frapv /tmp/P/casper /tmp/U
        time sudo cp -frapv /tmp/P/preseed /tmp/U
        time sudo cp -frapv /tmp/P/.disk /tmp/U
    

    The first copy above is of a 3500 MB directory named casper. This may take 5 to 20 minutes depending on the speed of your USBD.  Record the time outputs.

  5. [Install GRUB] Follow the GRUB installation section.

  6. [Verify that all needed files are present] Record the output of ls -lR /tmp/U

  7. [Done] Unmount your USBD.
        sudo umount /dev/sdb2

    USBD is ready to be used as a bootable device.  Leave the USBD in the PC.  Reboot the machine.  At the POST, press F12 for multi boot device choice.  Choose USB Device. Voila!

  8. [Use the USBD Linux Live] Boot from the USBD. Identify the full path names of (at least one each) (i) a word processor, (ii) a spread sheet, (iii) a slide presenter, (iv) a compiler for a PL of your choice, (v) a program development envirornment (IDE), (vi) a web browser, and (vii) a game. Put these names in the answers.txt file.

5.5 GRUB installation onto the USBD

This is step 5 from the Lab Experiment procedure placed here separately because of its importance. Do not carry this step out prior to mounting /tmp/U.

In this step, we are using legacy GRUB, not GRUB version 2, as the older version is simpler to setup.

  1. Copy GRUB files from the host machine:
    sudo cp -rapv ~ceg23300/grub /tmp/U

  2. The boot menu that GRUB will display is dictated by the file given by /tmp/U/grub/menu.lst.  Each "stanza" for this boot loader menu begins with the title-line.  The kernel line specifies the file to load as the kernel, and the parameters that GRUB should pass-on to the kernel.  The initrd line specifies the initial ram disk file.  This is expanded by the kernel during the first stage of the kernel boot.

    Save the following as a plain text file named /tmp/menu.lst.


    default 1
    color green/black yellow/blue
    
    title Booting via Grub from OSIS Lab Linux Live USBD 
    
    title OSIS Lab Linux Live USBD in Probed Graphical Mode
    kernel /casper/vmlinuz file=/cdrom/preseed/custom.seed boot=casper splash noprompt
    initrd /casper/initrd.gz
    
    title OSIS Lab Linux Live USBD in Safe Graphical Mode
    kernel /casper/vmlinuz file=/cdrom/preseed/custom.seed boot=casper xforcevesa splash
    initrd /casper/initrd.gz
    
    title OSIS Lab Linux Live USBD in Text Only Mode
    kernel /casper/vmlinuz file=/cdrom/preseed/custom.seed boot=casper textonly quiet
    initrd /casper/initrd.gz
    
    title Check the CD/DVD for defects
    kernel /casper/vmlinuz boot=casper integrity-check splash
    initrd /casper/initrd.gz
    
    title Non-Stop Test Memory until ESC key is pressed
    kernel /boot/memtest
    
    title Boot the First Hard Disk
    root (hd0)
    chainloader +1
    

  3. Copy the menu.lst file into the /tmp/U/grub directory:  
    sudo cp /tmp/menu.lst /tmp/U/grub/
     
  4. Please become familiar with what you should see in installing GRUB on the USBD before attempting this lab. On the first sign of unexpected output, stop and call one of the lab helpers to trouble shoot.

    GRUB presents itself below as an interactive shell with grub> prompt.  The rest of what is on the prompt is typed by me/you.  Output of GRUB is indented from the left margin by one space.  We inform GRUB that  the root of our OS installation is the second hard disk (hd1; GRUB counts starting from 0), partition 2 (the 1 after comma). 

    The find command is used solely to make sure that we are about to install the boot loader on the USBD and not the main hard disk; the root (hd1,1) you chose must match with the output of find.  Invoke Grub as:

    sudo grub

    The prompt by this program is "grub>". Note that there is a space after the first token in the commands to grub.

  5. The following is a record of a session I had with grub.   Your session may have a few differences. At the end, the "... succeeded" ought to appear; if not, some thing went wrong.
    Probing devices to guess BIOS drives. This may take a long time.
    grub> root (hd1,1)
    grub> find /grub/menu.lst
     (hd1,1)
    grub> setup (hd1)
     Checking if "/boot/grub/stage1" exists... no
     Checking if "/grub/stage1" exists... yes
     Checking if "/grub/stage2" exists... yes
     Checking if "/grub/e2fs_stage1_5" exists... yes
     Running "embed /grub/e2fs_stage1_5 (hd1)"... failed (this is not fatal)
     Running "embed /grub/e2fs_stage1_5 (hd1,1)"... failed (this is not fatal)
     Running "install /grub/stage1 (hd1) /grub/stage2 p /grub/menu.lst "... succeeded
    Done.
    
    grub> quit
    

    Note that in the dialog above we specify (hd1,1) as the root because we booted from the hard disk (which is therefore hd0).  When you chose the USBD as the boot device, it becomes hd0 and the hard disk that normally would be labeled hd0 now becomes hdN for some N > 0.


6. Trouble Shooting

This list is obviously very incomplete.

7. Turnin

  1. Note the number <n> of this Lab from the course home page and use L<n> as the first argument to turnin.  Turn in the files ReadMe.txt myLabJournal.txt answers.txt.
  2. Some of you may have already installed Linux on a USBD.  We would like to give you credit for that.   Explain in ReadMe.txt how you did it. Prepare as closely as possible the equivalent of myLabJournal.txt and turn them in.
  3. Keep the USBD with you, unmodified until we return the grades for this lab.  We may ask you to demo your USBD.

Link to Grading Sheet


Acknowledgements


References

  1. Boot and run Linux from a USB flash memory stick. http://pendrivelinux.com/ This site has many recipes, including ones that use Windows as the host OS, for installing different Linux distributions to a USBD.  Required visit.
  2. Ubuntu Linux OS http://www.ubuntu.com/. Linux is open source.  Debian Ubuntu Linux OS is free to download. It is legal to make as many copies as you want.  Community forum, wiki, etc.  Recommended visit.
  3. The LiveCD List http://www.frozentech.com/content/livecd.php  May not be exhaustive, but certainly lists more than a hundred.  Highly recommended visit.
  4. LiveDistro http://en.wikipedia.org/wiki/LiveDistro Required reading.
  5. Boot Sequence (i) http://www.pcguide.com/ref/mbsys/bios/bootSequence-c.html
    (ii) http://en.wikipedia.org/wiki/Booting (iii) Windows XP Boot Process http://dotnetjunkies.com/WebLog/unknownreference/articles/12284.aspx  Required reading.
  6. Prabhaker Mateti, Lab on Installing Linux on a One GB USB Drive, June 2009.  (Mainly for instructor's use.)
  7. What is an Operating System? http://en.wikipedia.org/wiki/Operating_system  Required reading.

Copyright © 2010 Prabhaker Mateti last edited: Dec 2010