START OF WEEK 1 of Linux
<<< FILE TIMESTAMP >>>
-The up to date file is in: /usr/courses/cps393/dwoit/courseNotes/
-If you are viewing your own copy, check it is up to date
-If you are viewing from a link in the Course Outline, be aware it may be outdated.
<<< UNIX/LINUX >>>
Linux [Unix] is an operating system (OS). So is Win
Many flavours: Ubuntu, Fedora, Debian, [Solaris, FreeBSD]
OSs based on Linux/Unix: MacOS, ChromeOS, Android, iOS
Widespread usage:
-86% of websites hosted on Linux/Unix servers
[http://w3techs.com/technologies/overview/operating_system/all]
-100% of supercomputers in world run Linux
-weather forecasting, code breaking, climate change modeling,
simulating designs for cars, aircraft, weapons, drugs,
health (Covid study, diagnosis, treatments)
-92% of top 25 websites in world use Linux
-96% of top 1million servers in world run Linux
[https://hostingtribunal.com/blog/linux-statistics/]
-Every person who uses the internet daily, probably uses Linux.
They may be unaware of it. [Linux.com]
[https://www.quora.com/Is-Linux-overrated]
-Linux is extremely underrated and people do not realize it is there half the time.
Your router uses linux, your ISP`s router uses linux, nearly all web servers
worth their salt run linux, datacenters run linux. Your smart-TV runs linux.
Your boardcomputer in your car runs linux. Your android phone runs on linux,
Your iPhone runs on UNIX-ish (very similar). Unless of course you are a bank:
[includes pic of ATM failure message saying Windows out of memory]
-Linux/Unix use includes:
-Netflix, TikTok, Instagram, Reddit, Tumblr, Twitter, Facebook
-Amazon, Google, Wikipedia, Samsung, Yahoo,
-Disney, Dreamworks, Pixar, E*Trade, Reuters,
-IBM, TiVO, DishNetwork, Garmin, US Department of Defense,
-Department of Energy, National Security Agency [Wiki.answers]
-Tesla displays, Toyota, Honda, Ford , smart TVs, drones, e-readers
[https://www.wired.com/2016/08/linux-took-web-now-taking-world/]
-Android OS is modified Linux kernel
-Travelocity, Cisco, Nokia, Sony, DejaNews, Match.com,
Mercedes-Benz, Boeing, HP, Compaq, etc.
-Microsoft (most Azure VMs are Linux)
[https://www.zdnet.com/article/linux-now-dominates-azure/]
-there are a lot more--google it!
-Stable and Secure:
[http://www.pcworld.com/article/204423/why_linux_beats_windows_for_servers.html]
OS: program(s) helping us communicate with computer`s resources
e.g., memory, processor, storage (e.g., to "run" a pgm--OS finds
in storage, loads into memory and loads 1st instruction into
program counter, and passes control to processor
OS is layered:
utilities: commands available (e.g. to copy files)
---------
shell: a pgm that reads our commands, figures out what
----- we want done, and gets it done
Win: shell called command interpreter (Command, PowerShell)
get it from GUI by start->command
prompts C:\>
reads
does
Linux/Unix: several shells available:
Bourne Shell (sh), Korn Shell (ksh),
C Shell (csh), Bourne Again Shell (bash)
prompts /home/dwoit>
reads, does
kernel: heart of OS
------
controls access to hardware, files
manages computer`s memory
maintains file system
allocates resources among various activities
(e.g. CPU time)
we can communicate with kernel through a shell through utilities:
e.g. /home/dwoit> cat myfile
display file named myfile on screen.
we do not care *how* data accessed from disk,
or *how* display works
we can use application programs:
e.g. browsers, games, music players
which communicate with kernel through utilities
moons:
-our Linux servers: metis, elara, europa, thebe
-our Unix servers: jupiter, vogon (solaris OS) -- but non-student
bash:
-we use command-line interface (shell window)
-reads commands, does them, displays results
Why learn CLI (command-line interface)? Why not just use GUI applications?
e.g., why learn to view/manipulate files in CLI instead of using a GUI
file manager (like file explorer on Win)?
-CLI is invaluable/necessary for DevOps
-efficiency. can save time/money.
-some tasks are efficiently completed through CLI but are too
complicated, expensive, or time-consuming through a GUI
e.g.,
-Climate change scientist needs to rename millions of files in a
15 TB filetree because Open Data naming conventions changed.
Could be done in seconds/minutes using CLI, but imagine the time
to do this using a file explorer GUI!
-Company needs to create 11,000 new client servers on Amazon Web Services,
all with identical configurations (except for 3 client-specific parameters).
Could be done in seconds/minutes using CLI to directly communicate
with AWS back-end. If used the AWS GUI server-creation interface, would
need to enter data into a form and submit for each of the 11,000
instances. And most of the fields will contain the same data for each
instance. CLI allows efficient automation.
-When GUI is appropriate, great, use it. But as good computer scientists,
we should have the ability bypass GUI when necessary for efficiency, and
we can do this using CLI.
-Once you know CLI, learning a GUI is easy. But not the reverse!
Logging into CLI (command-line interface):
Moon login: dwoit
Enter password: #does not appear when typed
/home/dwoit> #shell`s prompt (yours differs)
To login to a moon from LAB ROOM or remotely - see vimLab
For help, see TAs in lab, or ask at the CS Debug Room
https://www.torontomu.ca/cs/current-undergraduate-students/debug-room/
or see these resources:
https://bit.ly/2ZWNR7J
To Create/Edit files:
-use vim editor (see vimLab)
-may also use gvim, gedit, nano, emacs, or Xemacs
-Do not use any other editors, since only the ones above are
guaranteed to be available when you write the cps393 tests.
Pausing and Releasing terminal output:
-ctrl-s means pause terminal output (program keeps running)
IT FREEZES YOUR SCREEN
-ctrl-q means release terminal output (after being paused)
It unfreezes your screen
-can be useful if you have run-away output, but normally
causes windows-savy cps393 students grief because they
attempt to "save" their file on a moon by doing ctrl-s,
but this instead FREEZES your whole bash screen/window.
Everything they type is saved on a command queue, but it`s
not executed until you do ctrl-q.
If you`re a windows person, and are used to doing ctrl-s,
THEN REMEMBER ctrl-q since there`s no way to unfreeze your
screen without it. (Actually, there are other ways, but
they won`t make sense until next term.)
THESE NOTES ASSUME THAT WE ARE WORKING COMMAND-LINE,
IN A MOON SHELL WINDOW
(regardless of how you got there)
to logout of the moon:
/home/dwoit> exit
exit out of ALL your shell windows
On lab machines, ALSO logout of the local GUI!!! Usually
right-click and logout, or select from a menu.
<<< FILE SYSTEM (HOW OS ORGANIZES AND OPERATES ON FILES) >>>
Unix: commercial (like Win, iOS, macOS)
Linux: open-source distribution (Ubuntu, Android, etc)
Files and directories treated similarly
In these notes
-we may say "file" to mean "file or directory"
-we may say "entry" to mean "file or directory"
entry names:
CASE-SENSITIVE; 2 different files: MyFile myfile
max 255 characters (but system dependent)
no suffix needed e.g., myfile and myfile.jav both OK
File Organization:
files and directories (similar to Win)
file system like a tree (like Win)
top of tree is / (called "root")
/
|
--------------------------------
| | |
home bin usr
| | |
-------------- --------- courses
| | | | | | |
dwoit acari r2jan cp rm mv ----------
| | |
--------- bin cps393
| | | |
cps209 cps393 ------ --------
| | | | | |
oop --------- x86_64 sparc dwoit
| | | |
class.list labs ----------------- ----------
| | | | |
---------- submit-cps393dwoit labs courseNotes
| | | |
vimLab u1Lab --------------- ------
| | | | | |
-------- -------- u1Lab u2Lab vimLab u1.txt
| | |
vimLab.txt u1Lab.txt -------------
| |
vimLab.txt vimTutorial.txt
Moving around the filetree
working directory: the directory you are currently IN
-upon login, automatically IN your home directory
e.g., /home/dwoit
-change working directory using command: cd
e.g., cd /usr/courses
Specify entry name by its path through the tree (2 ways):
absolute path name: path starts at ROOT
e.g.,
to display vimLab.txt
cat /usr/courses/cps393/dwoit/labs/vimLab/vimLab.txt
relative path name: path starts at WORKING DIRECTORY
e.g., suppose IN /usr/courses/cps393 (working directory)
to display vimLab.txt:
cat dwoit/labs/vimLab/vimLab.txt
HMWK: For the file system (tree) given above, show how file
vimTutorial.txt can be displayed, using an absolute path name.
Show how file class.list can be displayed, using a relative path
name, assuming you are "in" directory /home/dwoit. What is the
absolute path name for directory usr? There are 2 "bin" directories
in the tree, give the absolute path for each. Give the relative
path for each, assuming you are in the root directory.
<<< LINUX COMMANDS >>>
-case-sensitive
-some basic ones:
ls, cd, pwd, cat, more, cp, rm, mkdir, rmdir, mv, wc, touch,
diff, tac, less, echo, printf, head, tail, rev, sort, uniq, tree
-comment: starts with # and goes to end of line
More specifically:
ls
lists contents of current directory (like dir in Win)
if in dwoit`s home directory:
/home/dwoit> ls
cps209 cps393
/home/dwoit> ls cps393/labs
vimLab u1Lab u2Lab
cd changes current directory (like cd in Win)
/home/dwoit> cd cps393
/home/dwoit/cps393> ls
class.list labs
/home/dwoit/cps393> cd /home/dwoit/cps393/labs
/home/dwoit/cps393/labs> ls
lab1.txt lab2.txt
note: . (this directory) .. (parent directory)
/home/dwoit/cps393> cd ..
/home/dwoit>
. . .
/home/dwoit/cps393/labs> cd #home directory
/home/dwoit>
/home/dwoit> cd cps393/labs
/home/dwoit/cps393/labs> cd ../..
/home/dwoit>
pwd (print working directory) shows you what directory you are in
(useful when shell environment does not display working
dir in prompt)
if in home dir and prompt is set to $>
$> pwd
/home/dwoit
$> cd ..
$> pwd
/home
cat displays contents of file(s) no paging
/home/dwoit> cat f1 #file f1 displayed on screen
this is line 1 of
the file
here is last line
/home/dwoit> cat fn1 fn2 fn3 #3 files displayed consecutively
tac cat lines in reverse order (last line to first line)
more like cat but paginated
(spacebar gets next screenfull, enter gets next line, q quits
less like more, but has extended commands/features
cp copies one file (or directory) to another
/home/dwoit/cps393> cp class.list new.class.list
/home/dwoit/cps393> ls
class.list new.class.list labs
/home/dwoit/cps393> cp /usr/courses/cps393/dwoit/labs/doc.txt .
rm deletes (removes) a file(s)
/home/dwoit/cps393> rm class.list new.class.list
mkdir creates a new directory
/home/dwoit/cps393> mkdir admin
/home/dwoit/cps393> ls
admin labs lab1.txt
rmdir deletes a directory (it must be empty)
/home/dwoit/cps393> rmdir admin
mv renames a file (or directory) "moves" it in filesystem
/home/dwoit/cps393/labs> mv lab1.txt lab1
/home/dwoit/cps393/labs> ls
lab1 lab2.txt
wc gives size (lines, words, chars) of files
useful to help count things
/home/dwoit/cps393/labs> wc u1M.txt
975 4349 33784 u1M.txt
diff shows difference between 2 files (prints nothing if identical)
home/dwoit> cat f1
abc def
ghi jk
home/dwoit> cat f2
abc def
ghi jkl
home/dwoit> diff f1 f2 #line 2 of f1 needs to 'c'hange in order
2c2 #to match line 2 of f2
< ghi jk #and shows the differing lines in each
---
> ghi jkl
home/dwoit> diff f1 f1
home/dwoit> #they are identical so no output
touch creates file(s) or updates modification time(s)
/home/dwoit> mkdir newdir
/home/dwoit> cd newdir
/home/dwoit> touch abc def ghi
/home/dwoit> ls
abc def ghi
echo prints the given string
/home/dwoit> echo hi there
hi there
/home/dwoit> echo "hi there"
hi there
printf similar to the usual printf
/home/dwoit> printf "hi %s %.2f\n" "there" 72.4
hi there 72.40
/home/dwoit>
head prints first 10 lines of given file
/home/dwoit> head myfile
first 10 lines of myfile printed
tail prints last 10 lines of given file
/home/dwoit> tail myfile
last 10 lines of myfile printed
rev prints each line in reverse (characterwise)
/home/dwoit> cat MF
line1
line2
/home/dwoit> rev MV
1enil
2enil
uniq prints each line, but omits duplicates
/home/dwoit> cat F2
zbc def
zbc def
zbc def
blue fish
blue fish
white mice
zbc def
/home/dwoit> uniq F2
zbc def
blue fish
white mice
zbc def
sort sorts input lines
/home/dwoit> sort F2
blue fish
blue fish
white mice
zbc def
zbc def
zbc def
zbc def
tree prints sideways tree rooted at given dir
/home/dwoit> tree /usr/courses/cps393/dwoit/labs/vimLab/
/usr/courses/cps393/dwoit/labs/vimLab/
├── ConnectToMoonRemotely.pdf
├── vim-files
│ ├── vim-cheat-sheet.txt
│ ├── vim-colors.txt
│ ├── vim-summary.txt
│ ├── web-ref-card.txt
│ └── web-tutorial.txt
├── vimLab.txt
└── vimTutorial.txt
# comments chars on line after #
/home/dwoit> touch f1 f2 f3 f4
/home/dwoit> ls f1 f2 #f3 f4
f1 f2
HMWK: give the sequence of commands that you would issue from your
home directory to do the following, in the order given: create 2
directories, lab1 and lab2, create a subdirectory of lab1 called
documentation; copy file /usr/courses/cps393/dwoit/labs/doc.txt
into your directory called documentation;
move your documentation/doc.txt into directory lab2 instead;
list the contents of directory documentation using an absolute path;
list contents of directory documentation again using a relative path;
make directory lab2 your current working directory; list contents of
file doc.txt; move back into your home directory; delete your file
doc.txt, then delete directory documentation, then, using an
absolute path, delete lab1 and lab2.
<<< COMMAND OPTIONS >>>
-change behavior of command
-specified following a - (dash)
-limits number of different cmds need to memorize
e.g.,
ls -t sorts newest to oldest
ls -R recursively lists all subdirs and files
ls -p print / following directory names
ls -l long list
/usr/courses/cps393/dwoit> ls -l
drwx------ 15 dwoit cps393 4096 Jul 8 14:25 archive
drwxr-xr-x 5 dwoit cps393 4096 Feb 29 11:25 courseNotes
drwxr-xr-x 14 dwoit cps393 4096 Jul 8 15:16 labs
diff -q X Y if files differ, print brief message instead of
default output (all the differences)
diff -s X Y if files identical, print message instead of nothing
diff -sq X Y prints only brief message saying identical or differ
echo -n "hi there" does not print the newline at end
echo -e "hi\tthere\nagain" enable backslash escapes
hi there
again
try the above echo without the -e
head -4 X prints only first 4 lines of file X
tail -28 X prints last 28 lines of file X
sort -n X numeric sort (vs text sort)
rm -r labs ("recursively" erases whole sub-tree rooted at labs)
-can combine. Order usually irrelevant. Same:
ls -l -t
ls -t -l
ls -lt
ls -tl
e.g.,
/usr/courses/cps393/dwoit> ls -tl
drwxr-xr-x 14 dwoit cps393 4096 Jul 8 15:16 labs
drwx------ 15 dwoit cps393 4096 Jul 8 14:25 archive
drwxr-xr-x 5 dwoit cps393 4096 Feb 29 11:25 courseNotes
<<< COMMAND-LINE MANUAL >>>
man basic manual pages
/home/dwoit> man ls #the man pages for "ls" command
#are "more"d on your screen
/home/dwoit> man -k zip #keyword/apropos search for "zip"
/home/dwoit> man -k rename #keyword/apropos search for "rename"
info /home/dwoit> info mv #the info pages for "mv" command
#more friendly? emacs-like
info --vi-keys #for vi-like movements
many online, e.g.,
https://linux.die.net/man/
https://www.gnu.org/software/bash/manual/html_node/
But tests will only have man in bash shell
HMWK: From your home directory, create 3 files, called tst1, tst2
and tst3, with vim (just enter one word or sentence in each file).
Use the man pages to determine the option for the "ls" command that
will get the following displayed on the screen: tst1, tst2, tst2
Use the man pages to determine the option of the "more" command that
will display a group of consecutive blank lines as a single blank
line on the screen.
Use the man`s apropos option to find all the browsers supported on linux
<<< LINUX SECURITY >>>
each user:
userid, password, home directory /home/userid/
may login many times simultaneously
normally only userid may manipulate files/dirs under /home/userid
but userid may give other users various permissions too
<< OWNERS AND GROUPS >>
each file and dir has an owner and a group associated with it
Owner: -when you create a file, you become its owner (usually)
Group: -users can join "groups" of other users with whom they
can share files & dirs
-users can join many groups, but have one primary group
-users can "work" in different groups (change groups with
newgrp newgroup)
-when a user creates a file, the file`s group is made to
be the group the user was in when the file was created
(for you, typically your primary group)
ABOVE INFORMATION ABOUT GROUPS HOLDS FOR LINUX IN GENERAL; HOWEVER,
IN CS, ONLY SYSADMINS CAN CREATE GROUPS AND ADD USERS TO GROUPS.
<< PERMISSIONS >>
rwx (read/write/execute) for each of ugo (user/group/other)
note "user" means entry "owner"
permissions on a file/dir indicate:
-how owner, group members and other users may operate on it
for each file/dir:
-each of user (u), group members (g), and other users (o) have
a set of read (r), write (w), and execute (x) permissions
(in that order)
r: file: can be displayed, copied
dir: can look at its contents (list dir`s entries )
But cannot display contents of a file inside dir (requires
r on the file and x on dir for that)
w: file: can modify or delete file
dir: add/del entries to/from dir
Note can modify a file in dir without w on dir (if w on file)
x: file: can run it, if it is a runnable (executable) file
e.g., /usr/bin/ncal
dir: weaker than r. Can access an entry of dir, if know its name
but cannot list contents of dir
e.g., typically, public_html has only x for other. Thus,
anyone may display/read index.html (and any file/dir
whose name they know), but cannot list contents of
public_html to see what else it contains
Permissions Examples:
wchan wants anyone who knows the name and location of
/home/wchan/labs/myfile to be able to execute it,
but does not want anyone to be able to copy it, see its
contents, delete it
Perms:
-exactly --x for other on myfile
-at least --x for other on all dirs down filetree to myfile
so anyone can GET to it: /home, /home/wchan, /home/wchan/labs
wchan wants http://www.cs.torontomu.ca/wchan to display her
index.html, but does not want anybody on moons to be able to
discover what other files she has in public_html, nor
add/delete files to/from her public_html
Perms:
-at least r-- for other on index.html
-exactly --x for other on /home/wchan/public_html
-at least --x for other on /home, /home/wchan
dmason wants to keep all his files/dirs private
Perms:
-group and other --- on /home/dmason
kavi wants no other student to be able to steal her
assignment 1 file, which professor said must be named
Assignment_1 It is located in her home directory.
Perms:
-Assignment_1 has --- for group/other
dwoit accidentally removes w perms for user on /home/dwoit
So what happens?
She can still modify files that already exist in /home/dwoit
(as long as they have rw perms for user) but cannot delete
the files, nor add any new ones to /home/dwoit
file /home/kavi/myfile has -w- for user.
kavi can change the file; however, kavi CANNOT read it.
How can kavi change the contents of myfile?
Not with an editor, like vim because vim must read it first.
Can still "change" it other ways like the linux commands:
cp /home/wchan/public_html/index.html myfile
echo "new stuff" >>myfile #meaning of >> is below
<< VIEWING/CHANGING PERMISSIONS >>
Can see perms with command: ls -l
-rwxr-xr-- 1 dwoit cps393ta 1734 Aug 19 2024 myFile
^\ /\ /\ / ^ ^ ^ ^ ^ ^
| | | | | | | | | file name
| | | | | | | | last modification date
| | | | | | | size in bytes
| | | | | | group name
| | | | | owner
| | | | number of links to file
| | | permissions for other users (only read here)
| | permissions for members of group (only read and execute here)
| permissions for owner of this file (read, write and execute here)
file or directory? - = file, d = directory
for each of user, group, other, there are 3 possible permissions
(read,write,execute)
if read is set, "r" in first position, otherwise "-"
if write is set, "w" in second position, otherwise "-"
if execute is set, "x" in third position, otherwise "-"
In example above, owner has read, write and execute perms on myFile,
group has only read and execute perms on it
all others have only read perms on it
only owner can change permissions on entries
use chmod command in Symbolic or Numeric Mode.
Symbolic Mode:
chmod (ugoa)(+-=)(rwx) name(s)
= changes to exactly those given perms
+/- add, remove given perms
e.g.
suppose myFile starts with perms -rwxr-xr--
chmod g+w myFile -rwxrwxr--
chmod ug-x myFile -rw-rw-r--
chmod a+r+w+x myFile -rwxrwxrwx
chmod go=rx myFile -rwxr-xr-x
chmod g=x,o+w myFile -rwx--xrwx
Numeric Mode:
- - - (000) 0
- - x (001) 1
- w - (010) 2
- w x (011) 3
r - - (100) 4
r - x (101) 5
r w - (110) 6
r w x (111) 7
chmod 160 myFile sets only x for user, only rw for group
and no perms for others
vvvvvvvvvvvvvvv optional vvvvvvvvvvvv
| umask sets up default permissions for all subsequently created files/dirs
| tells which perms to EXCLUDE
| e.g., umask 022 exclude write perms for group and other
| do not exclude any perms from user
| (excluding from what was set up as default
| by sysadmin--typically rw for user)
|
^^^^^^^^^^^^^^^ optional ^^^^^^^^^^^^
HMWK: If you had a directory called dog that had read permissions
for everyone, and write permissions only for the owner, and execute
permissions for the group and others, what would the permissions
part of a "long listing" of the directory look like? Assuming you
were in the parent directory of dog, what is the command that would
remove all permissions for group and other? Then, what command would
give execute permissions to everyone, read permissions to the group,
and remove write permissions for the user?
You have default permissions for all files you create. If you have not
done so yet, create 3 files, called tst1, tst2 and tst3, with vim
(just enter one word or sentence in each file). Then from your home
directory, use whatever commands you need to discover what the default
permissions are on all files you create.
Express this in numeric mode: chmod u=rwx,g=rw,o=x myfile
HMWK:
Create a file called tst1, and a directory called dog, in your
home directory (if you have not done so already.) Copy tst1 to a
file called cat in directory dog. Move into dog. Remove write
and execute permissions for user (owner) on file cat. Now
try to erase cat. What happens? Why? How can you erase it? (give
2 different ways, if possible)
<<< INPUT/OUTPUT (I/O) STREAMS >>>
Linux commands, and pgms, do I/O e.g.,
ls *writes* results to screen (output);
passwd *writes* a messages to screen (asking for old/new passwords, etc)
passwd *reads* your old/new passwords from the keyboard
How does shell know WHERE to read/write to?
The shell associates *streams* with the command.
a stream
is a flow of data
like a tunnel (data in one end and out the other)
data flows through a stream in one direction only
The shell assigns 3 standard streams to any command:
stdin, stdout, stderr
The command may then:
read data from stdin
write data to stdout
write error messages to stderr
e.g.,
stdin stdout
---------------------- ------------------------
data in --> passwd data out -->
---------------------- ------------------------
| |
| | stderr
The shell normally attaches other end of the streams to devices,
such as the keyboard and monitor
e.g.,
stdin | | stdout
---------------------- ------------------------
keyboard passwd monitor
---------------------- ------------------------
| |
| | stderr
monitor
when passwd wants input (e.g., the current password)
reads from stdin == reading from keyboard
(passwd does not know or care that keyboard is attached to
stdin, just reads from stdin)
Output similar
when passwd encounters error, writes error msg on stderr
(it ends up appearing on monitor, but passwd does not
know or care)
NOTE: many commands ignore stdin if you give them a file to operate on.
e.g., cat lab1 #ignores stdin & reads directly from lab1 instead
e.g., wc lab1 #similarly
Try these with no arguments. What are stdin, stdout, and stderr?
(note ctrl-d closes stdin once you`re finished typing)
wc
rev
cat
<<< I/O REDIRECTION >>>
You can tell shell to redirect any std stream
/home/dwoit> ls > myfile #redirects stdout
shell attaches other end of stdout to file myfile
to see output of ls command, must view contents of myfile (vim, more, cat etc)
> myfile clobbers (over-writes) myfile
>> myfile appends to end of myfile
/home/dwoit> wc < myfile #redirects stdin
e.g., if cat cmd is given no argument, stdin is keyboard
/home/dwoit> cat >outfile
/home/dwoit> my first line
/home/dwoit> my second line
/home/dwoit> ^d #Linux end of file/stdin (^z in Win)
#NOT ^z. It does something else in Linux
now file outfile contains:
my first line
my second line
/home/dwoit> cat <infile >outfile
/home/dwoit> #nothing displayed
/home/dwoit> cat <outfile
#file outfile displayed on screen
#now has same contents as infile
#these have same result
/home/dwoit> tail -37 infile >lastPart
/home/dwoit> tail -37 <infile >lastPart
#file lastPart contains a copy of the
#last 37 lines of infile
The diff command allows - for stdin so this diffs file XYZ and stdin
/home/dwoit> diff XYZ -
whatever
^d #end of file (stdin)
#diff gives no output if XYZ contains only "whatever",
#and shows differences, otherwise
to redirect stderr use "2>"
e.g., Suppose labs does not exist. Then error msg on stderr:
/home/dwoit> ls labs
ls: cannot access 'labs': No such file or directory
/home/dwoit>
If redirect stderr, error msgs written elsewhere:
/home/dwoit> ls labs 2>errfile
/home/dwoit>
/home/dwoit> cat errfile
ls: cannot access 'labs': No such file or directory
Use redirect and /dev/null to throw output away
/home/dwoit> ls labs 2>/dev/null
/home/dwoit>
Aside: why 2> ??
each stream given a number by shell
stdin 0, stdout 1, stderr 2 (ls >outfile is same as ls 1>outfile)
(cat <infile is same as cat 0<infile)
To redirect stderr to stdout:
2>&1
to redirect both stdout and stderr at once
&>outfile
or
>outfile 2>&1
<<< DEVICES >>>
device: usually a piece of equipment for storing or
communicating data,
e.g., printer, disk drive, terminal, modem
on Linux, we access devices as if they were (special) files,
typically in dir /dev
e.g., a printer might be "file" /dev/lp0
therefore cp myfile /dev/lp0
prints myfile on line printer
e.g., writing a message directly to a terminal:
echo "hi there" >/dev/pts/4
stdin/stdout/stderr are
/dev/stdin, /dev/stdout, /dev/stderr
displaying file contents on stderr could be done:
cp blah /dev/stderr
bash assigns stdin/stdout/stderr to file descriptor 0/1/2
so
echo blah blah >&2 #writes to stderr
these both redirect stdout:
echo blah >file
echo blah 1>file
these both redirect stdin:
cat <file
cat 0<file
when OS recognizes an operation on a special file, it calls
a pgm called a device driver to do the op (e.g., cause data
to be displayed on screen)
Null Device (/dev/null) the garbage can, black hole -- data
can go in but never come back out.
sending output to /dev/null = throwing it away
sometimes commands produce output we want to disregard
e.g., get error msgs if some files do not exist
/home/dwoit> wc file1 file3 file4 file5 file6
wc file1 file3 file4 file5 file6
7 8 34 file1
wc: 'file3': No such file or directory
wc: 'file4': No such file or directory
28 32 136 file5
21 24 102 file6
56 64 272 total
Throw away only the errors:
/home/dwoit> wc file1 file3 file4 file5 file6 2>/dev/null
7 8 34 file1
28 32 136 file5
21 24 102 file6
56 64 272 total
--more examples later
HMWK: Copy file tst1 to file tst.new, without using the cp command;
instead use output redirection.
Make sure there is no file called xxx in your home dir. Using the
"cp" command, try to copy file xxx to file yyy. What happens? Is
there a way you can use output redirection so that "cp" will not
display any messages on the screen when you try to copy xxx?
What commands could you use to create a file called dir.and.list
that contained the name of your current directory on the first
line, and a list of files in the current directory on subsequent
lines (you must create dir.and.list using Linux commands, *not*
using an editor, such as vim.)
Show 2 different ways you can use the "cat" command to create a
file called big.tst, which contains file tst1, followed by tst2,
followed by tst3.
The nl command can be used to number lines given on stdin. Try it.
Use the nl command to make a file named num1.txt, which is an exact
copy of unix1.txt, except with line numbers. You must get nl to read
unix1.txt by REDIRECTING stdin.
END OF WEEK 1 (Linux)
May do u1Lab now