This tutorial shows you how to use Linux file system events (notify) to be notified whenever a file is displayed in a directory. You can use these as triggers to automate common tasks on your system.
We are going to write a script that looks at a directory and works on new files that are added. Each file is sent and moved to another directory as soon as it is detected. The script uses the inotify subsystem, via a tool called inotify-tools. But first, let's install the tool and experiment.
Installing inotify tools and gzip
apt-get to install this package on your system if you use Ubuntu or other Debian based distribution.
sudo apt-get install inotify-tools gzip
Experimenting with inotify tools
Let's start by looking at a directory and seeing which events begin when new files arrive. We will use a tool called
inotifywatch that is part of the inotify tool. Create a new directory called "incoming":
Begin looking at this directory by performing the following command:
inotifywatch -v incoming
This will instruct inotify to look at all file system events in "incoming" directory. The
-v option enables the tool to print extra information about what it does. We have not specified a timeout option
(- t ), and the command will continue to collect events until we end with CTRL + C. At this time, our terminal should look like this:
Open a new terminal window (or tab) and switch to the incoming directory. Use the command command to create a new file named "newfile".
cd incoming / press newfile
Now go back to the first terminal window and stop inotifywatch by dialing CTRL + C.
A table of events will be served to the console, indicating an example of "create", "" Attrib, "and" close_write. "These four events occurred when we used the contact to create a new file, enter its file access attribute, opened it to write a blank character, and then shut it down. These are just some of the many events that can be monitored on a file system with inotify- You can see the full list on the main page of inotifywatch.
For our purposes we are only interested in two events:
- "create" – when a file is created in the target directory.
- "moved_to" – when a file move from another location to the target directory.
Let's try to inotifywatch again, but this time it instructs to just monitor these two events. Run this command in the first terminal window:
inotifywatch -v -e create -e moved_to incoming
In the second terminal window or tab let's try to create a new file, change the content and then move the file from another location to the target directory. ndon runs from the home directory.
touches incoming / created echo testing123 >> incoming / created touch / tmp / created2 etc. / tmp / created2 incoming /
Return to the first terminal window and stop inotifywatch by dialing CTRL + C. We see the following output:
Only two events were counted: create a file called "created.txt" and move an existing file called "created2.txt". Everything, like changing "created.txt", was ignored.
Look at a directory and perform a task
Now that we know which events to follow, we can use another tool called
inotifywait to block the run until a file is created in or moved to our target directory. We use the same arguments as we did with inotifywatch and also indicate how we want the file name to be formatted for use in our task.
Before we start, we need a directory to keep files that have already been processed. Create a directory called "processed":
mkdir is processed
Then create a new script called "watch-incoming.sh" and add the content below:
#! / Bin / bash TARGET = ~ / incoming / WORKING = ~ / processed / inotifywait -m -e create -e moved_to - format "% f" $ TARGET | while reading FILENAME do echo Detected $ FILENAME, moves and zips "$ TARGET / $ FILENAME" "$ PROCESSED / $ FILENAME" gzip "$ PROCESSED / $ FILENAME" Done The script executes the inotifywait command with the option
-m. This makes the monitoring command change indefinitely. Each time a new event is detected, the file name is sent to the read command and injected into the variable "FILENAME". The block below the hose is executed, where the file is first moved to the "processed" directory and then gzipped. The original file is replaced with the compressed file, and the file name will end in ".gz".
Let's grant driving permissions on this script and run it from our home directory.chmod u + x incoming. sh ./watch-incoming.sh
Open the second terminal window and create a new file in the "incoming" directory. List the contents of both the "incoming" and "processed" directory to see the results of the event being detected:
The raw text file we copied to the "incoming" directory was detected by scripts, copied to "processed" and then compressed with gzip.
We can do some interesting tasks now that we can look at new files that come in a directory. For example, we can add a watermark to image files, compress raw videos to mp4 format and even upload every new file we look to an Amazon S3 bucket. This script is a good starting point for rolling your own workflows and automating common tasks on your system.