Chapter 17 - Searching for files

Picking back up at ,“A return to the playground. “ The first exercise in this section is creating a playground doing some searching inside of it. These are the commands used to create the playground

#to make the directories
mkdir -p playground/dir-{001..100}

#to make the files 
touch playground/dir-{001..100}/file-{A..Z}

#we can make a timestamp for a directory by putting an empty file in it with touch and then stating that file. 
touch playground/timestamp
stat playground/timestamp

#you can update the timestamp by using touch on that file again

most of the time exec is called in reference to find we need to call it with the brackets and semicolon escaped like so ‘{}’ ‘;’. In the man pages it said you may benefit from -exec + in some cases Im not really sure why though.

I am sure now. With the semicolon find executes each instance of the command individually and with the + it does them all together in one command with lots of argumetns.

locate, updatedb, find, and xargs are part of the GNU findutils package provided for you. Most modern linux distributions remove locate and updatedb and replace them with newer and better packages such as plocated which are usually included in a seperate package.

Chapter 18 - Archiving and Backup

One of the system admins primar y tasks is keeping the system’s data secure and backed up. There are a few methods of doing this we are going to talk about

data compression is the idea that you can remove redundancy from data and still perserve the integretity of the data. This is most easily exhibited with an all black photo. We dont need to send the all black photo to tell the receiver that the photo is all black, so there are ways we could compress the photo.

Two different types of compression

• lossless - all the data contained within the orignal will be contained in the new when uncompressed.
• lossy - you lose some data to allow for more compression and for the data to be more easily sent. It doesnt match the orignal one to one but is a farily good approximation of what it would have looked like.

gzip

gzip is one of the tools we can use to compress and uncompress files. Its sister program gunzip is used to unzip and uncompress files from gzip. When unzipping you dont have the specify if the files end with gz the program assumes that they do.

bzip2

is similar to gzip but uses a different compression algorithm. Acheives higher levels of compression at the cost of compression speed. bzip2 also comes with a program called bzip2recover which will try to recover a bzip2 file.

A lot of files are already compressed so dont try to run compression algorithms on them as usually that will only make the files larger. Some formats that are already compressed include the following:

Rule of thumb

If the format’s name or spec already implies a codec or archive algorithm, running gzip on it just adds overhead.

Formats not in this table — BMP, uncompressed TIFF, PPM/PGM/PBM, WAV, AIFF, and raw text — are the ones where compression can still save space.

Archiving Files

archiving is the process of gathering many files and putting them in one large file. Often done as a part of system backups.

tar usually operates with the following syntax

tar mode[options] pathname

here are a few modes

• c - create an archive
• x - extract an archive
• r - append specified pathnames to the end of an archive. If the archive does not exist create it.
• t - list the contents of an archive.

its possible to only extract a certain file from a tar. This is done by specifying the pathname and then extracting it. The files in a tar can be listed before you extract them so you can know what you want to extract before you do it as well. Normally when specifying paths in tars wildcards are not supported however in the GNU implementation which is the implementation most often seen by people on linux distributions you can specify globbing with the –wildcards option.

Modern implementations of tar support gzip or bzip2 compression directly with the z and j options. The filename ‘-’ can be used with tar to mean standard input if we are trying to tar the result of a command like find. tgz is the standard for gzip compressed tar files.

if you are windows user zip is usally how you copmress and decompress files. However on linux gzip is usually the standard.

zip it up

when using zip there a few things to note with the output. The stored % output is the amount of compression that zip acheived on that run

  adding: playground/dir-100/file-V (stored 0%)
  adding: playground/dir-100/file-W (stored 0%)
  adding: playground/dir-100/file-X (stored 0%)
  adding: playground/dir-100/file-Y (stored 0%)
  adding: playground/dir-100/file-Z (stored 0%)
  adding: playground/timestamp (stored 0%)

in this instance zip acheived 0% compression. Since our playground contained empty file no compression was done. Fun little thing with piping

yes | unzip ../playground.zip

Its possible to pipe input into zip using the -@ command.

rsync

a common strategy for maintaining backups of a system involes keeping one or more directories synchronized with another. for example you may have a local copy of a website under development and want to synchronize it from time to time with the “live” copy. Which is just the actual website. rsync is the preferred tool for doing this task.

rsync can quickly detect the differences in two directories and update only the files that changed. If we want to make rsync compatable over a network you can specify it to use ssh like so.

sudo rsync -av --delete --rsh=ssh /etc /home /usr/local remote-sys:/backup

This will tell r shell to use ssh.

Instead of doing this you can also use an rsync daemon which will listen for incoming request for synchronization and respond to them accordingly. This is often done to allow mirroring of a remote system. An example of how this would be useful is for software repositories for fedora or other os distribution maintainers. Instead of making backups multiple times a day they have a rsync daemon keep a few mirros in tact so they are all the same. So if anything happens to one of them there are still a few others with hopefully mostly up to date information. If you wanted to sync with the development tools repository mirror for fedora kept at duke university you could use this command.

mkdir fedora-devel
rsync -av -delete rsync://archive.linux.duke.edu/fedora/linux/development/rawhide/Everything/x86_64/os/ fedora-devel

Something important to note here is the protocol specification at the beginning of the URI. It’s rsync!

break

I will come back to rsync above I marked the page in the bookmarks file. I wanted to do a tryhackme module real quick though.

Linux Room 3

You can use python to serve a http.server that comes prepackaged with python 3. You can set up the server like so

python3 -m http.server

you can use this to server any sort of file from your computer onto to any machine that you are connected to with wget. -m is the module-name. So you are running the python3 module http.server. By default this module will serve whatever directory it is run in. However you can also point it at a directory like so. In addition to that I specified the port number here.

python3 -m http.server 8000 --directory /path/to/dir

ps aux and top are good monitoring tools for your system for processes. top being live ps aux being one time call.

I used a grep search with regex to be able to find the flags for one try hack me challenges I just did. This was the command

ps aux | grep -G .*{.*}.* | less

we use systemctl to stop and start services available on the machine. like so

systemctl start | stop | enable | disable | status  service

we would use the enable command to have the service start on boot. That is what it means by enable.

cron

a crontab is a special file with formatting that is recognized by cron process to execute each line step-by-step. Crontabs require 6 specific values

If we don’t care what month day or year that a command is executed in then you just put an asterisk in that field.

Quick Note For Andrew it doesnt immiediatley appear as if cron is on my cachy machine. Ill have to do some digging to see if there is a more modern replacement for modern linux machines. Or maybe cachy just prefers a different job runner.

This crontab creator website seemed useful. im going to link it here.

Crontab Creator

Windows server operating systems exist. Teh current one is Winodws Server 2025.

Windows Room 1

action center is apparently where the notification center sits. not really sure how that is going to be useful.

the ntfs filesystem is short for new technology file system. Before NTFS for windows there was FAT16/FAT32 and HPFS. ntfs is known as a journaliing file system. That means it can automatically repiar folders/files on disk using information stored in a log file. The function is not possible with FAT. NTFS can notably set specific permission on folders and files, do folder and file compression, and encrypt filesystems using EFS.

• this link to a documentation page for FAT, NTFS, and HPFS will probably be pretty good. link to fs doc

You can view perms for a file or folder in windows using the properties tab. It sits in the secuirty tab. Alternate Data Streams is another feature of NTFS. Every file has a dat stream. powershell can be used to view data streams, that will be covered in a future section.

a desciption of environment variables and their format %envvar%. Description:

• “ Environment variables store information about the operating system environment. This information includes details such as the operating system path, the number of processors used by the operating system, and the location of temporary folders “.

Accidentally deleting folders in system32 can you make your os inoperational.

%windir% is the system variable for a windows folder. To see other users on a windows system you can type in the start menu other user and then go to system settings > other users which will allow you a place to manage multiple users on a system. You can use lusrmgr.msc from the run prompt if you want to do the same thing.

Changing the icons in control panel will bring up more settings you can look at and control. That is a bullshit feature.

Windows Room 2

System configuration uitlity or MSConfig is for advanced troubleshooting. Main purpose is to help diagnose startup issues. You may need admin rights to open this utility. You can specify different types of startups in this utility so you can isolate the problem.

Microsoft advises using Task Manager to manage (enable/disable) startup items. System config utility is not a startup manager. Meaning they don’t want you to use it for that.

On windows server machines you will not see startup programs in Task Manager or msconfig. The only reliable way is to use the run menu and then type

shell:startup

Windows uses a page file when physical RAM becomes full. You can view and modfiy it by navigating to advanced at the setting menu under performance.

fuck windows room 2. That was some bullshit fr.

Chapter 19 - Regular Expressions

not all regular expressions are the same. For much of the use cases in the book we will be using regular expressions as defined by the posix standard. I wanted to include a table of a few other falvors though.

grep usually accepts options and arguments in the format below

grep [options] regex [file...]

a few notable options listed by the book include

• –invert-match - prints lines that dont match the pattern rather than do
• –files-with-matches - prints files that contain matches rather than any text that does.
• –quiet or –silent - suppresses all output. the book was saying this is useful for seeing if a match was found at all with shell scripting and piping multiple commands together.

in addition to literal characters which are characters that just match to themselves, regexes also match to metacharacters which are used to specify more complex matches. Meta characters are very similar to globbing and wildcard matching from shell languages.

• ‘.’ - is used to match any character

the caret and dollar sign cause a match to occur only at the beginning or end of a line.