Useful Ubuntu Linux Commands

Record Audio and Video from a Webcam to File

The order of the video and audio inputs appears to matter here - if the audio is put first, then in my case at least, the video was recorded with about 2 seconds of audio lag. If you want to see what is being recorded while it's happening, the following worked for me: This assumes you are using pipewire (or pulseaudio I guess) and that the name of the audio sink you want to record from is called alsa_input.pci-0000_00_1f.3.analog-stereo which it probably isn't. View the audio sinks on your system with:

View Details of Opened SSH Tunnels

You can use the lsof (list open files) tool to see what SSH tunnels are open on the current machine. Pass it the -i flag to have it list all internet files (i.e. sockets), and optionally filter the results with grep to just get SSH-related ones: lsof automatically replaces IP addresses with hostnames - if you don't want it to that (e.g. if host-name lookup is slow for some reason), then also pass it the -n flag.

Kill All Open SSH Connections

Sometimes temporary network failure causes my SSH connections to hang indefinitely e.g. if I'm connected to patchy WiFi or I'm using a mobile hotspot in a bad signal area. This behaviour is probably related to a settings change I made recently to prevent my SSH connections being dropped too easily, but until I can find a better solution, the following will just kill all open SSH connections:

Skip Certain Directories in find

Sometimes when running find, you don't want to look in certain directories e.g. Composer's vendor dir, or Subversion's .svn dir. Here's how you can do that:

Copy All Modified Files in an SVN Repo over FTP

This method uses cURL to do the transfer, as it allows us to do it in a single line. We also use the -I argument for xargs to specify a substitution string for the input strings. Run this from the root of your local repo: Note that this will upload all files that appear in the SVN status output - not only new and modified files but also files with conflicts, so run svn st first to ensure only files you want to upload are listed.

How to Search within Results for Files Containing a Second Search String

The following command uses find combined with grep via the exec argument to search all PHP files for a search string, and then feeds the names of matching files to grep to search within them for a second string, using xargs:

Fix PHP Files with Windows-1252 charset saved as UTF-8

During a large-scale customer site migration, some files which had been saved in Windows 1252 encoding were moved and saved as UTF-8 without conversion. This led to characters such as the apostrophe, em-dash and ellipsis appearing as black diamonds when served through Apache. iconv can be used to convert them into proper UTF-8 encoding like so: You'll need to change /srv/http above to the root of the site you want to convert. Also, because iconv doesn't have a 'convert in place' option, each converted file is saved with a .utf8 extension, and then moved on top of the original file. To be on the safe side, a backup of the original file is taken first, and saved with the extension .php.old.php.

Filter a list of domains that resolve to a given IP

Given a list of domain names in the file /tmp/doms, one per line, the following command will output only those which resolve to the IP specified:

Delete All Subversion Branches under a Specified Folder

This process can be automated by using svn rm in its second mode - deleting a URL - as this can be performed in a single step with a commit message specified on the command line. It's necessary to remove the trailing slash which svn ls appends using bash's split string operator otherwise svn rm won't recognise the argument as a URL:

Remove all SVN branches that have been merged to trunk

Buidling on the previous command, the following adds a mergeinfo check to only remove branches that have merges onto trunk:

Delete files Older than a Certain Age

This command will delete all files in the current directory that have not been modified in the last 30 days: Note the use of -not -name to exclude hidden files. If you want to test what it's going to splat before it does it (always a good idea), run the same command but using the ls action in place of delete:

Pipe the Output of a text file into find

The problem with piping stuff into find is that the path to search is often not the last item on the command line. xargs and its replace option can help. With the replace option, you use the -i switch to specify a name that xargs then replaces with each line of the incoming input like so:

Find the Most Recently Modified File in a Directory Tree

First we use find to get all files which are not directories and output their UTC timestamp followed by their full path. This output is piped into sort using the -r switch to sort in reverse order, then passed into head with -1 to just give us the very first name produced: Use the -path switch along with -prune to tell find to ignore certain filepaths: Note the use of the -o and -a logical operators to join expressions for find.

List the Targets of All Symlinks under a Particular Directory

Simply use find with the -exec option: Note the empty curly braces which denote where the filenames returned by find should be used, and also the final, escaped semicolon which denotes the end of the -exec command.

Remove File Associations created by WINE

Get rid of all the associations with the following: If you want to just remove one or two associations, list the directory contents, and the file names make it quite clear which file deals with which extension.

Recursively Copy Read-Only Files while Preserving File Attributes

Sometimes you might find yourself needing to copy a bunch of files organised in a directory tree from one place to another, keeping the files' positions in the tree intact along with their file attributes. If the files are not read-only, this is a simple matter of copying or moving the parent folder to the destination location. If the read-only bit is set, it's not so easy, and the following combination of find, rm and mv will do the job:

How to Change X org Screen Resolution

Use xrandr for this. Run it without arguments to see what resolutions are supported by the attached monitor. Then run it again using the -s switch to specify the resolution you want:

How to Test the First Character of a String on the CLI

The ${VARNAME::1} syntax will give you the first character in $VARNAME. For instance, say you want to list all files and directories that begin with a digit in the current directory: Notice we're using the =~ operator to do a RegEx match, against the POSIX [:digit:] character class. The double colon and curly brace notation ${VARNAME::1} to get the first character is in fact just a specific usage of the bash substring operator. For instance, if you want to find the first 3 characters of a variable, simply change the 1 to a 3 and use ${VARNAME::3}. This is in fact equivalent to ${VARNAME:0:3} meaning 'get the 3 characters of $VARNAME beginning at offset 0'. Thus, you can find the 4th and 5th characters of a variable with ${VARNAME:3:2} (NB character 4 is at offset 3 because of zero-based counting).

How to Split a String In bash

There are two operators that we can use to split a string on a given delimiter character, % and #. They are used in conjunction with the curly bracket operators to get the portion of a string before the delimiter (%), or the portion after (#). For instance, you could split filenames into the main part and the extension like this: Note that the second example, which gets the extension, uses a double ##, which means 'split and return the shortest possible substring', while a single one would return the longest possible substring. Obviously it only makes a difference when there's more than one occurence of the 'split character'.

Set Page Up and Down keys to Search bash History

Create a file in your home directory called .inputrc containing the following lines: Now type the first few letters of a command, press Page Up and bash will show you the last command you entered that began with those letters. Press again for the previous match, and so on.

Increase the Sudo Timeout

After you run a terminal command as root by prepending it with sudo, you can execute further commands with sudo without reentering your password as long as each successive command is no more than 5 minutes apart, by default. You can change this default by adding a line like that below to your sudoers file: By setting the timeout to -1 as above means that there is no timeout, and you only need enter your root password the first time you run sudo in any terminal session. Alternatively, use a positive integer to specify how many minutes you want the timeout to be. Don't forget that the right way to edit your sudoers file is by using visudo.

Recursively Applying chmod To Files Only

There are often occasions when you need to ensure all files in the current folder and all subfolders have specific access attributes, but chmod's -R flag would also apply the same attributes to directories. This is usually a problem because directories need the executable bit set if they are to be accessible. The solution is to use find in combination with chmod: Use -type d if you want to apply chmod only to directories rather than only to files.

Handy Bash Keyboard Shortcuts

Ctrl + L	Clears the Screen, similar to the clear command
Ctrl + Z	Puts whatever you are running into a suspended background process. fg restores it.
Ctrl + W	Delete the word before the cursor
Ctrl + U	Clears the line before the cursor position. If you are at the end of the line, clears the entire line.
Ctrl + K	Clear the line after the cursor
Ctrl + T	Swap the last two characters before the cursor
Esc + T	Swap the last two words before the cursor
Ctrl + _	Undo
Alt + .	Insert last word from history at the cursor

Clean Deadwood out of SVN Repo

It's not uncommon for extraneous files to find their way into a subversion repository over time. Backups, test scripts, IDE setting files and folders; all these and more can be found washed up in your once pristine repos. The more people you have in your team, the more likely this is to happen, and the more varied the deadwood flies will be. Often there will be multiple files with a similar filename scattered throughout, such as the subfolders called _notes that dreamweaver creates inside every folder.

We can splam all these in one go with a little CLI Fu. This bash command will remove all _notes folders from subversion, and then delete them from disk: Obviously, you'll need to do a commit afterwards, and you'll probably want to do an update immediately beforehand.

Create a TAR archive with a Folder Tree starting at the Current Working Directory

Use the anchored option as below:

Find files in a directory that are mentioned within files in another directory

This command lists all the files inside the directory called srcdir, and then searches all files in targetdir, including those inside subdirectories, looking for any that contain the filenames from srcdir. It outputs just the names of files from srcdir that are mentioned within other files, along with a count of how many times each was found.

This is useful when trying to determine if any files in a directory are no longer in use, ie no longer linked to from (or included by) other files.

Get File From an arbitrary SVN Repo

Get SVN History Since a Specific Revision

This will retrieve all commit messages, dates and users for all commits from revision 31 inclusive. You can substitute another revision number in place of HEAD if you want to limit to a range of revisions.

List All Files Modified Since a Specific SVN Revision

Basically use the same SVN command as above, but with the verbose switch and then feed into grep, sort and uniq to get a list of what files were modfiied, added or deleted:

Find SVN Revisions Committed by a Specific User

Do this by passing the SVN log through grep: This will just show details such as the date, time and revision number from the commit message. You could use the -A2 switch on grep to view the first line of the commit message for each change, but this will take up screen space of course, and can easily eat up all your scrollback if this is not a new project and has a lot of history in the log. Alternatively you can just view the comment for a specific commit by replacing the XXX below with the revision number: SVN also has the -lNN option which will limit the response to NN lines, but of course this is before it's further filtered by grep.

Find a server's external IP address via DNS from CLI

Read a file and perform a command for each line

The following trivial example simply copies a file from inputfile to outputfile, but naturally you can replace the echo command with any other of your choosing.

Useful Additions to .bashrc

Cool Stuff to put in ~/.inputrc

.inputrc holds config options for readline, the bash command line editor.

Set Disk Checking to Once per Month

Ubuntu defaults to running a disk check every 30 boots, or once a month (I think). The following command will set it to just check once per month, regardless of how many times you reboot:

Search within Selected Files Recursing Through Folders

grep has the -r option to do a recursive search through folders under the specified path. However it won't work in conjunction with wildcards such as *.java. To do searches such as this, you need to use it in combination with find and xargs like so:

Create a tar.gz Compressed Archive

The -c option means create a new archive, -v is for verbose, -z indicates it should be a gzipped archive, -p means preserve permissions on files, and -f means use the specified archive file.

List contents of a tar.gz Compressed Archive

Unzip a ZIP archive

List All directories containing certain files

This will list all directories under the current directory which contain python files.

Flush bash history

The bash shell keeps a record of commands that have been used in the file ~/.bash_history. However the commands used in the current session are not written to this file until you log out of the current session. If you wish to flush the history to this file before logging out, use the following: 'a' here stands for append - so this is the regular behaviour. If you want, you can use 'w' instead, which will replace the current bash history file with the session history.

Exit a bash Session without Saving History

If you don't want to leave certain sensitive commands in the bash history file, just exit your session with the following: This leaves all the previous history intact. If you have accidentally placed some sensitive information in the history file, you can remove the sensitive info using sed like so: The -i option stands for 'in place', and you need to specify it when you want sed to replace strings in a file, as opposed to, for instance, piping the altered file elsewhere. If you alter the contents of the bash history file as above, don't forget to exit your session using the preceding commands so you don't leave the sed command containing the sensitive string in history.

List the most recently modified file

Switch to a Virtual Terminal from the command line

Display a Unix timestamp in a readable form

Reset WINE (equivalent to a reboot in Windows)

Restart the pulseaudio deamon

Restart Apache2 on Ubuntu

View Apache2 error log

This location is also where apache keeps its access.log file.

Run Disk Check on Next Reboot

Sometimes you might find that having a file named forcefsck in the root directory will make fsck run a check on the next boot: For me though, this doesn't seem to work very reliably. In such cases, adding fsck.mode=force to the kernel line in Grub does work, and a check will be performed on every available partition.

Reload Apache2 Config Settings

Install Apache2 Module (eg ModRewrite)

sudo a2enmod module_name

eg sudo a2enmod rewrite

Then force Apache to reload config settings as above.

Regenerate the fonts cache:

The default location of system fonts is /usr/share/fonts/. Different font types are grouped into subfolders off here.

Create a symbolic link

ln -s /path/to/target /path/to/linkname
You can omit the final linkname argument if you want to create a link in the current directory with the same name as the target directory.

Change the target of an existing symbolic link

ln -sf /path/to/target /path/to/linkname
The 'f' option (or --force) actually overwrites any existing file with the same name & location as the newly created link. If not given, it should just hide any such files until the link is destroyed.

The effect is that if the link already exists, this command will change it so it links to the specified target location. If it doesn't already exist, it will be created as normal.

Copy a file over a network via SSH

ssh -l {login_name} 192.168.0.XXX cat /path/to/target > /path/to/outputfile

Shortcuts for Eclipse IDE

Lower Case: CTRL+SHIFT+Y
Upper Case: CTRL+SHIFT+X

Jump to Matching Bracket: CTRL+SHIFT+P

Copy current line(s) and place them above/below: CTRL+ALT+UP/DOWN
Move current line(s): ALT+UP/DOWN

Comment/uncomment current line(s): CTRL+/

Switch to next/prev tab: CTRL+PgUp/PgDn

Close current file: CTRL+W

Find open file: CTRL+E

Delete to end of line: CTRL+SHIFT+DELETE
Delete previous word: CTRL+BACKSPACE

Import Gzipped MySQL Database on the command line

gunzip < /path/to/database.sql.gz | mysql -u root -p -D databasename

The final -D databasename option for the mysql client selects the named database, and is only necessary when the gzipped SQL file doesn't begin with a USE databasename command.

Unix if conditional single-line syntax

The key to rememeber is that commands must be separated either by a newline or a semi-colon, and that if, then and fi are all commands.

Split a File into Many Files each containing one line from the first

That heading pretty much describes it. Say for instance you have a list of hostnames in a text file called /tmp/hostlist, you can run it through the following command to generate a list of consecutively-numbered files each containing one hostname:

Unix for loop syntax

The following returns every line in all files called build.mk in all subdirectories of the current working directory which contain the text 'bird', along with the name of each file: Typical output might be:

./src/mozilla/extensions/build.mk:# Ben Turner mozilla@bongbirdnest.com

The name and path of each file appears, a colon then the actual line containing the string 'bird'.

Mount an ISO Image in Unix

sudo mount -o loop /path/to/iso /path/to/mount/point

Replace Underscores with Spaces in File and Directory Names

List installed packages

How you do this will depend on the package manager used by your distribution. For RPM based distros, such as Red Hat, Fedora Core, Suse Linux, Cent OS, use:

rpm -qa | grep -i 'package-name'

The i switch on grep is pretty unnecessary really, as all libraries seem to use only lower case.

For .deb systems such as Debian and Ubuntu, use:

dpkg --list | grep -i 'package-name'