The objectives of this lab experiment are to make you :
This lab will install a Linux distribution that can bee booted and run entirely from a USB drive. The Linux system that is so-installed is a full-scale system, suitable for all the CS/CEG courses of WSU. This lab is done early in the term so that all the students have "their own" Linux that they can run on home PCs without disturbing Windows or other software.
You will understand processes better in a couple of weeks.
Unless a PC has disabled booting from a USBD, this lab lets you boot from the USB drive and become the privileged user named root.
An OS controls all the hardware of a computer, and every program currently running. All computer systems, by definition, include an OS. Several modern day gadgets, such as cell phones, MP3 players, wrist watches, PDAs, video game consoles, and TiVO, are specialized computer systems. Many appliances, such as microwave ovens, dishwashers and TVs, contain embedded computer systems.
Currently popular OS for personal computers include Windows, Linux, MacOS, and Solaris. Our discussion in this article is so modern that software known as CP/M (1960s), MS-DOS (1960s) will not qualify as OS.
A computer system can be divided into three pieces:
A typical PC fits this classification very well; but, there are many classes of computers that do not. Some computer systems may not have a key board or a hard disk.
A program is a sequence of CPU instructions (also called machine code) structured in a very specific way. Software is a generic synonym for programs. Applications are a subclass of software.
The CPU can execute its instructions only when they are in RAM. We have so many programs that it is infeasible to leave them all sitting in RAM. The OS stores them on HDD and loads them into RAM as needed.
We will explain in the lectures the "Fetch, Decode, and Execute cycle" of a CPU.
CPU can generate traps and faults.
Devices raise interrupts. Interrupt handling is done by the OS.
The moment a computer system is powered on, the firmware is in control of every piece of the hardware. On a PC, this firmware is called the BIOS (basic input output subsystem).
A computer system must not be shut down "ungracefully". All the running processes need to be informed of the impending shut down, and wait politely for them to terminate. This is known as graceful termination of the OS. Most systems can now detect a power failure, and initiate a shutdown. However, un-responding processes not withstanding a shutdown cannot be postponed indefinitely, and the OS forcibly terminates.
There are many definitions of OS based on its typical functionality. From the book by Silbershatz and others: "What are the three main purposes of an operating system? Answer: (i) To provide an environment for a computer user to execute programs on computer hardware in a convenient and efficient manner. (ii) To allocate the separate resources of the computer as needed to solve the problem given. The allocation process should be as fair and efficient as possible. (iii) As a control program it serves two major functions: (a) supervision of the execution of user programs to prevent errors and improper use of the computer, and (b) management of the operation and control of I/O devices."
A very useful definition of an OS is based on its components. If we were to spread out all the code that constitutes an OS on a piece of cloth, we can then cut it up into the following components:
OS = VM + FileM + ProcM + Net + IO + UserM + (Boot) + [SysPrg]
Strictly speaking, an OS does not include GUI, nor CLI. In the world of Linux, the GUI is provided by X11, Gnome and KDE software subsystems, and CLI by bash, csh, zsh or other such shells. These are not strictly part of the Linux OS. In the Windows world, the GUI is provided by (plain) Explorer, and the CLI by cmd or PowerShell. These are not strictly part of the Windows OS.
Programs like Word, Internet Explorer, Konqueror are applications. Visual Studio, g++, Active Python and such are (sub-) categorized as development tools.
The above components can also be combined and then re-divided into a kernel + initial ram disk components + drivers + interrupt handlers.
The code of system calls is not disjoint from the code of previously mentioned components. It is made of different pieces from these components. Linux and Windows have around 350 system calls each. The details of Linux syscalls can be found in the files:
-rw-r--r-- 1 root root 1403 2010-01-03 06:57 /usr/include/sys/syscall.h -rw-r--r-- 1 root root 22041 2010-01-28 00:56 /usr/include/asm/unistd_64.h
/bin, /usr/bin, ...: These directories do contain programs, but these are not considered part of the OS. The programs in these directories are considered simply applications.
Most of the OS stays RAM resident and dormant until (i) a system call is made by a process, (ii) a hardware interrupt causes a handler in the OS to run, or (iii) a software triggered even causes a handler in the OS to run. Consequently, viewing an OS only as a collection of processes is going to be incomplete.
|1||?||Ss||0:00||/sbin/init||The First Process|
|3||?||S<||0:00||[migration/0]||Processes shown within brackets are part of the Kernel.|
||Dynamic Device Manager service|
|1053||?||Ss||0:00||portmap||RPC port map service|
|1056||?||Ss||0:00||dd bs=1 if=/proc/kmsg ...||Kernel message logging|
|1088||?||Ss||0:10||hald||Hardware Abstraction Layer|
|1357||?||S<||0:00||[nfsiod]||NFS Serivice IO|
|1479||tty4||Ss+||0:00||/sbin/getty||User login process|
|2738||?||Ss||0:00||/usr/sbin/nmbd||Samba Name Service|
|2748||?||Ss||0:00||/usr/sbin/smbd||Samba File Service|
We will explain in the lectures: Boot sequence. POST. Locating bootable media. Loading the MBR. Loading the OS boot loader.1. Powered on. BIOS->POST. ROM and RAM. CPU fetch-decode-execute cycle.
Ubuntu and other distributions use UUID (Universally Unique Identifier) to indicate the root device. From the Wiki: In its canonical form, a UUID consists of 32 hexadecimal digits, displayed in 5 groups separated by hyphens, for a total of 36 characters. For example: 550e8400-e29b-41d4-a716-446655440000 .
See Required Reading below.
There are way too many (about a thousand?) distributions. No one has a complete list, but the DistroWatch site is quite extensive. The "major" distributions are (in no particular order): Debian (Ubuntu), RedHat, SuSE, Gentoo, and SlackWare. Most of the minor distributions are specializations of these.
We are using an ISO customized for OSIS Lab usage. It is about 3.5 GB in size. Its lineage is: osislablinuxlive.iso ⇒ Linux Mint ⇒ Kubuntu Linux ⇒ Debian.
Windows 3.1, 95, 98 are from one code lineage. The NT, 2000, XP, Vista and Windows 7 are from a different lineage.
There are two ways of installing an OS onto a USB mass storage drive (USBD). This assumes that your laptop/desktop is recent enough to permit booting from a USBD.
Insert the USBD and start the OS installation. During the install, select the USBD as the target drive. Modern OS installation procedures will discover the USBD as if it is a normal HDD, and let you partition, format, and mount the partitions. This has the advantage of higher speed, but requires a larger USBD.
Boot into a host OS. It can be Linux or Windows. Transfer the content of an OS LiveCD onto the USBD. Then, install an OS boot loader on the USBD. Most live CDs have their larger files compressed, and before they can be used must be uncompressed, either before hand or on the fly as needed. So, this does not have the same speed as an installation done as in the above. But, the required size capacity of the USBD is no more than that of the live CD. Knoppix Live CD compresses about 2 GB worth of programs onto a 700+ MB CD. There are live CDs ranging from 50 MB to 760 MB. There are live CDs of many OS: Linux, FreeDOS, FreeBSD, OpenBSD, Solaris, and even Windows. There are now (2010) over a hundred live DVDs.
In this lab, we will be following method 2, using Linux as the host OS. You will be installing a Linux distribution on your own USBD, which should be at least 1 GB in size. Be aware that all its content will be destroyed during this lab. (Note that some (old) USBD are known to have hardware issues when used as bootable devices.)
In this lab, you will be invoking some commands that are
privileged because they have a destructive possibility. You are
expected to look up the man pages of
sudo, fdisk, mkfs.ext2,
grub. Be extremely
careful in what arguments you supply to these commands; double-check
before you press return.
This section is not a step-by-step procedure. It explains the behind the scenes reasoning of the steps you will carry out later on. Peculiarly perhaps we are not just interested in creating a bootable USBD; we are more interested in understanding the internal stucture of what directories and files such a USBD has.
[Partitioning the USBD] All mass storage devices can be viewed as a long sequence of sectors/blocks. These devices can read/write in units of blocks only; they cannot read/write single bytes. A consecutive section of blocks is a partition. A large hard disk can be divided into several partitions. A small disk will be made into single partition. There are primary, extended and logical partitions. For simplicitly, we divide the USBD into two primaries.
We make a VFAT volume on the first partition of the USBD so that the USBD can be readily used in Windows also.
Linux and hence the partitioning program "think" that the USBD is a HDD, so the use of terms like cylinders etc. should be discounted.
The partions on a 4 GB USBD should look like the following. Your numbers may not match exactly with these. A block is 1 kB.
Disk /dev/sdb: 4009 MB, 4009754624 bytes 126 heads, 22 sectors/track, 2825 cylinders Units = cylinders of 2772 * 512 = 1419264 bytes Disk identifier: 0x000a8f3b Device Boot Start End Blocks Id System /dev/sdb1 1 283 392227 c W95 FAT32 (LBA) /dev/sdb2 284 2825 3523212 83 Linux
[Making File Volumes on USBD] Before any thing can be stored on a partition, it needs to be initialized. This is known as making a file volume. Formatting is a lower level activity, but in the Windows world this often means making a file volume.
[Mounting ] To access the files on a volume, it must be "mounted." Note that files are persistent on a USBD or HDD, but are not "on-line" until mounted.
[Copy files of LinuxOS] Typical commands that you are familiar
with such as ls, mkdir, bash, and the KDE etc. are all
packed into one file named
can crudely describe this as a zip archive. More precisely, it
is the compressed file-system image containing many files. The Linux
kernel consists of two files:
vmlinuz and the initial
[GRUB Installation] This step if carried with the slightest of the typos can destroy the hard-disk booting of our lab PCs. We are using GRUB as our OS loader.
The OS-on-USBD you produce at the end of this Lab is will work like any other hard disk installed Linux. There are several limitations of course: (i) Due to the smaller size of the USBD compared to typical HDD, the range of installed software is limited. (ii) The speed suffers: (ii-a) USBD are several times slower than HDD, and hence. (ii-b) Also, there may not be a swap partition. (ii-c) On the USBD, we use a compressed read-only file system that is uncompressed piece meal as needed. (iii) Because it is read-only, new or updated software cannot be installed.
Once you boot from the USBD, there may be couple of dialogs about keyboard, time zone, etc. depending on the ISO image we chose.
Some motherboards of desktops/laptops have trouble booting (hangs
or reboots) unless
acpi is turned off, etc. So,
experiment with the kernel parameters by interacting with GRUB.
As you shutdown, there may be a dialog about other virtual terminals being active. Just ignore this.
Our procedure in creating this USBD mimics a LiveCD. So, there may be a dialog about removing "the CD" and then pressing Enter. Just press Enter. This can be eliminated by appending the token "noprompt" after the token "splash" to the grub menu stanzas.
Our USBD, the way we constructed it, does not save any customization of appearances, etc. that you may make. So, on the next boot from this USBD these are all lost.
After booting from this USBD, you can mount the first (vfat) partition, and save your work there.
All work is expected, but not required, to be carried out in the Operating Systems and Internet Security (OSIS) Lab, 429 Russ. But, you are welcome to work wherever. Note that use of both Linux and Windows and other software, that may not always be installed in other facilities, may be needed.
It is expected that ypu have understood the previous sections, especially the Overview.
The USBD size expected depends on the ISO image we are using. It is best to have at least a 4 GB USBD. It can be much bigger (untested in the TB range).
During this Lab, accept no pop up dialogs that offer to auto-mount the USB drive.
Do all the following in a Linux
"Record" below means copying-and-pasting appropriate lines (e.g., (i)
a command and (ii) its output) that appear in this window into a plain
text file named
myLabJournal.txt and also in
[Partitioning the USBD] Insert your USBD. In the OSIS lab, its
device name will be
/dev/sdb. If a popup dialog asks you
about mounting, do not click on OK; click on Do-Nothing. When doing
this experiment elsewhere the name of the USBD may be
different. If you use the wrong name, you will be wiping out
what was on the (wrongly) named device. Invoke:
Delete all existing partitions on your drive. Press
and type the number of the partition to be deleted. Do this for all
We are going to make two primary partitions. Make the first partition taking up all the space on the USB leaving 3500+ MB for Linux.
You type n for new,
p for primary, 1 for
partition number, then we specify the needed size.
The size specification can be confusing mostly because we see "cylinders" and this is a USBD. Accept the default start cylinder of 1. Specify the size using the syntax of "+sizeM", that is, the plus-sign, followed by the needed size as a number, then a big M, all without any spaces in them. E.g., "+390M" (without the double quotes).
Make the second partition to be a 3500+ MB primary partition for
n for new,
p for primary,
2 for partition number, and accept the defaults.
Make sure that this partition is 3500+ MB in size. In the partition
example above, this second partition came out with 3523212 blocks. If
not, redo by going back to deleting all the partitions, and choosing a
smaller first partition.
Set the type of the first partition to vfat. Type
c (or 6 etc).
Record a listing of these partitions. Type
print details of the partitions.
Write to the USBD the internal data structures that the above
steps have created:
w for write.
[Making File Volumes on USBD] This will take a few seconds.
vfatfile volume on the Windows partition.
sudo /sbin/mkfs.vfat /dev/sdb1
Create an ext2 file volume on the Linux partition:
sudo /sbin/mkfs.ext2 -L casper-rw /dev/sdb2
Record the output of
Mount OS image:
mkdir -p /tmp/P sudo mount -o loop,ro /var/ISO/osislablinuxlive.iso /tmp/PThis is the source image of the Linux OS image we are installing.
Mount your USBD:
Record the output of
mkdir -p /tmp/U
sudo mount /dev/sdb2 /tmp/U
time sudo cp -frapv /tmp/P/casper /tmp/U time sudo cp -frapv /tmp/P/preseed /tmp/U time sudo cp -frapv /tmp/P/.disk /tmp/U
The first copy above is of a 3500 MB directory named casper. This
may take 5 to 20 minutes depending on the speed of your USBD.
[Install GRUB] Follow the GRUB installation section.
[Verify that all needed files are present] Record the output of
ls -lR /tmp/U
[Done] Unmount your USBD.
sudo umount /dev/sdb2
USBD is ready to be used as a bootable device. Leave the USBD in the PC. Reboot the machine. At the POST, press F12 for multi boot device choice. Choose USB Device. Voila!
This is step 5 from the Lab Experiment procedure placed here
separately because of its importance. Do not carry this step out
prior to mounting
In this step, we are using legacy GRUB, not GRUB version 2, as the older version is simpler to setup.
sudo cp -rapv ~ceg23300/grub /tmp/U
/tmp/U/grub/menu.lst. Each "stanza" for this boot loader menu begins with the
kernelline specifies the file to load as the kernel, and the parameters that GRUB should pass-on to the kernel. The
initrdline specifies the initial ram disk file. This is expanded by the kernel during the first stage of the kernel boot.
Save the following as a plain text file named
default 1 color green/black yellow/blue title Booting via Grub from OSIS Lab Linux Live USBD title OSIS Lab Linux Live USBD in Probed Graphical Mode kernel /casper/vmlinuz file=/cdrom/preseed/custom.seed boot=casper splash noprompt initrd /casper/initrd.gz title OSIS Lab Linux Live USBD in Safe Graphical Mode kernel /casper/vmlinuz file=/cdrom/preseed/custom.seed boot=casper xforcevesa splash initrd /casper/initrd.gz title OSIS Lab Linux Live USBD in Text Only Mode kernel /casper/vmlinuz file=/cdrom/preseed/custom.seed boot=casper textonly quiet initrd /casper/initrd.gz title Check the CD/DVD for defects kernel /casper/vmlinuz boot=casper integrity-check splash initrd /casper/initrd.gz title Non-Stop Test Memory until ESC key is pressed kernel /boot/memtest title Boot the First Hard Disk root (hd0) chainloader +1
menu.lstfile into the
sudo cp /tmp/menu.lst /tmp/U/grub/
Please become familiar with what you should see in installing GRUB
on the USBD before attempting this lab. On the first sign of
unexpected output, stop and call one of the lab helpers to trouble
GRUB presents itself below as an interactive shell with
grub> prompt. The rest of what is on the prompt
is typed by me/you. Output of GRUB is indented from the left
margin by one space. We inform GRUB that the root of our
OS installation is the second hard disk (hd1; GRUB counts starting
from 0), partition 2 (the 1 after comma).
find command is used solely to make sure that we
are about to install the boot loader on the USBD and not the main
hard disk; the
(hd1,1) you chose
must match with the output of
The prompt by this program is "grub>". Note that there is a space after the first token in the commands to grub.
grub. Your session may have a few differences. At the end, the "... succeeded" ought to appear; if not, some thing went wrong.
Probing devices to guess BIOS drives. This may take a long time. grub> root (hd1,1) grub> find /grub/menu.lst (hd1,1) grub> setup (hd1) Checking if "/boot/grub/stage1" exists... no Checking if "/grub/stage1" exists... yes Checking if "/grub/stage2" exists... yes Checking if "/grub/e2fs_stage1_5" exists... yes Running "embed /grub/e2fs_stage1_5 (hd1)"... failed (this is not fatal) Running "embed /grub/e2fs_stage1_5 (hd1,1)"... failed (this is not fatal) Running "install /grub/stage1 (hd1) /grub/stage2 p /grub/menu.lst "... succeeded Done. grub> quit
Note that in the dialog above we specify
the root because we booted from the hard disk (which is therefore
hd0). When you chose the USBD as the boot device,
hd0 and the hard disk that normally would be
hd0 now becomes hdN for some N > 0.
Symptom: Takes a long while, and finally you see an "initramfs" prompt. Cause: The ext2 volume made must have a label of casper-rw, but does not. Fix: Redo the lab starting from Section 5 step 2.
Symptom: Does not recognize USBD as a bootable device. Cause: Grub enabling on the USB might have problems. Fix: It is too hard to verify. Instead, just redo the Grub Installation above.
Symptoms: Varied. Overall, cannot boot into Linux Live. Cause: Not all USBs are "good". Many develop bad sectors (i.e., when we write data D in them and read it back some time later we get some thing different from D) over time. Not a fix, but Linux command "badblocks" can search a device for bad blocks.
Symptom: Windows sees only the first partion. Cause: By Windows policy, I guess. Fix: If you wish to get rid of the two partitions on the USBD, and bring it back to factory default condition, use fdisk to delete the two partitions, make one FAT32 partition occupying the entire device, and write the new partion table back to the device. Invoke mkfs.vfat on that partition.
turnin. Turn in the files
ReadMe.txt myLabJournal.txt answers.txt.
ReadMe.txthow you did it. Prepare as closely as possible the equivalent of
myLabJournal.txtand turn them in.
Link to Grading Sheet