Windows File Integrity Checker like sha1sum on Linux

Why doesn't Windows have a built in file integrity checker like sha1sum on Linux. You'd think this would be pretty important - for example when verifying the integrity of Windows and Microsoft product updates.

There is a tool you can download called:

Microsoft File Checksum Integrity Verifier

I see that it has no provided checksum :) Of course what would you use to check it? You'd possibly have to pull out your Linux machine. (not funny?)

http://www.microsoft.com/en-us/download/details.aspx?id=11533 

To check a file:

1. Download the app.
2. Extract to desired folder
3. Add that folder to your PATH environment variable (search on this if you don't know how)
4. Open new command prompt window.
5. Navigate to file you want to check
6. Type fciv.exe to get output with all options
7. Type this to get sha1 hash of your desired file

fciv.exe <file> -sha1

8. Compare that hash you just created to the hash provided by the site where you downloaded the software

I noticed there was a -v option to verify a file and compare to it's hash. Seems to require an XML file which I tried but failed to read my file so I just compared the two manually since it was a short value. Would be better if they had a tool that worked more like sha1sum on linux, that takes a text file with the key as input.

Linux Notes

Various Linux commands in no particular order
_______________________________________

Coming soon...

Cron jobs

Hostname

Addition in bash, spacing

su

Any commands that require admin privileges can be run by a user not logged in as root by putting su in front of the command. Best practice is don't log in as root and use su. You'll need to enter a root password. 

_______________________________________

sudo

Put in front of any single command to run that command as root without entering the root password.

Users who can sudo are listed in /etc/sudo

Edit with visudo 

http://aplawrence.com/Basics/sudo.html
_______________________________________

List what's in current directory

ls
_______________________________________

Output file to screen in terminal window

Originally cat was use for concatenating files, but many people use it just to output the file to the screen

cat [file]
_______________________________________

sed, awk, tr, cut - text formatting and manipulation
 
sed and awk are stream processors. Pass in a string and get transformed output. scripting.

tr will take input and replace or delete characters in the new output

cut extracts a portion of a file by selecting columns

_______________________________________

Display particular lines of file (head and tail)

Show beginning lines of file

head file

Show last lines of file

tail file

Show lines 10 - 20 of file

head -10 $line file | tail -20

or awk and sed

http://unix.stackexchange.com/questions/47407/cat-line-x-to-line-y-on-a-huge-file


_______________________________________

Tee

Output results of a command to standard out and file at the same time

cat [file1] | tee [file2]
 
http://linux.101hacks.com/unix/tee-command-examples/
_______________________________________

Find a file or application

In current directory:
 
find -name "file name in quotes"

From root
 
find / -name "file name in quotes" 

Ignore case:

find -iname "file name in quotes"

Using wild card (e.g. find all files that end in .conf)

find -name "*.conf"

Find all files with permission (e.g. 0777)

find -type f -perm 0777

Use locate

locate "*.jpg"

More:

https://www.digitalocean.com/community/tutorials/how-to-use-find-and-locate-to-search-for-files-on-a-linux-vps

_______________________________________

Find Text in a File

To find the string "text" in any text file:

grep "text" *.txt

Use strings to read the human readable portion of a binary file

strings [file]
_______________________________________

Show permissions of files in a directory

ls -l

or

ls -al

or recursively display permissions of subdirectories

ls -lR
_______________________________________

See Permissions for Specific File

ls -l [file name]
_______________________________________

Find all the files owned by a particular user

find / -user [user]
_______________________________________

Find all the files owned by a particular group

find / -group [group] 
_______________________________________

File & Directory Permissions

When file permissions are listed look something like this:

-rwxrw-r-- [user] [group] [file]

Each of first 10 characters have following meaning:

d if directory otherwise -
r if owner can read otherwise -
w if owner can write otherwise -
x if owner can execute otherwise -

r if group can read otherwise -
w if group can write otherwise -
x if group can execute otherwise -

r if everyone can read otherwise -
w if everyone can write otherwise -
x if everyone can execute otherwise -

This is followed by owner, group and file name.
_______________________________________

Numerical Permissions

The above 10 character permissions can be translated to numbers. The shortened, numerical form of permissions is used by most Unix admins when setting permissions. A letter = on (1) and a dash = off (0).

Chars = binary = decimal
rwx = 111= 7
rw- = 110 = 6
r-x = 101 = 5
r-- = 100 = 4

So to give full rights to owner (7), read only to group (4) and no rights to everyone (0) would be 740

More on binary, hexadecimal and binary if you really want to know
http://websitenotebook.blogspot.com/2014/05/hexadecimal-to-binary-to-decimal-cheat.html
_______________________________________

List users on system

awk -F':' '{print $1}' /etc/passwd

Or 

cat /etc/passwd

_______________________________________

Determine user home directory

cat /etc/passwd

_______________________________________
 

Login as another user

su [user name]

_______________________________________

List groups on system

cut -d: -f1 /etc/group

_______________________________________

View contents of file that defines groups

cat /etc/group

_______________________________________ 

Add a user to a group

usermod -a -G <group> <username>

_______________________________________

See if user has root permissions by checking to see if they have UID 0

grep 'x:0:' /etc/passwd
_______________________________________
 
See which users are in root group

grep root /etc/group
_______________________________________

See which users can execute as root

cat /etc/sudoers
_______________________________________

List programs installed

compgen -c
_______________________________________

Find a specific program

compgen -c | grep bash
_______________________________________

Check which programs can execute as root using SUID bit

find / -perm -04000
_______________________________________

Edit Linux networking

Edit the file specific to the interface you want to alter

/etc/sysconfig/network-scripts/ifcfg-eth0
_______________________________________

View Firewall Rules

iptables -L -n
_______________________________________
 
Stop Firewall

service iptables stop
_______________________________________
 
Create Firewall Rules

iptables(8) -A INPUT -m state --state ESTABLSHIED, RELATED --j ACCEPT

Typically accept state ESTABLISHED, RELATED and NEW. Drop others.

http://explainshell.com/explain?cmd=iptables+-A+INPUT+-m+state+--state+ESTABLISHED%2CRELATED+-j+ACCEPT

http://explainshell.com/explain/8/iptables 

http://security.stackexchange.com/questions/4603/tips-for-a-secure-iptables-config-to-defend-from-attacks-client-side

http://ipset.netfilter.org/iptables.man.html 
_______________________________________

Distinction between "shell" and "terminal emulator"

The shell handles commands. The terminal or terminal emulator provides a way to send commands to the shell from a graphical user interface.
_______________________________________

Change default terminal

update-alternatives --config x-terminal-emulator

_______________________________________

TERM environment variable

The TERM environment variable is updated by the terminal, not used to define it.

_______________________________________

Change file permissions 

chmod [permissions] [file]

Common file permissions 

777 no restrictions (no good)

755 owner can write, anyone can read & exec

700 only owner has rights and can read, write, exec

666 all users may read and write

644 owner may write, anyone can read

600 owner may read and write


Directory permissions

777 no restrictions

755 owner has full access, others can list files, not create or delete

700 directory owner has full access

_______________________________________

Create a file

Use a text editor

or

echo "some text to put into a file" > [file]
_______________________________________

Delete a file

rm [file]
_______________________________________

Change file owner

chown [user] [file]
_______________________________________

Change file group ownership 

chgrp [group] [file]
_______________________________________

How to tell if a process is running

ps aux | grep [process]
_______________________________________

So to see if SSH is running

ps aux | grep sshd
_______________________________________

To see network information including mac and IP

ifconfig
_______________________________________

To see gateways and routing information

route -n

Or

netstat -r 
_______________________________________

Default gateway on Linux 

The last entry in the route table using defines the default gateway.

route -n

More:

http://www.idevelopment.info/data/Networking/Networking_Tips/ROUTERS_Gateways_Routing_Table.shtml

http://www.cyberciti.biz/faq/linux-setup-default-gateway-with-route-command/

http://linuxconfig.org/configuring-default-gateway-on-redhat-enterprise-linux 

Multiple interfaces (see last answer):
 
http://serverfault.com/questions/597791/linux-adds-the-wrong-default-route
_______________________________________
List Services

ls /etc/init.d
_______________________________________

View Running Services

top

or

ps -Al 
_______________________________________

Generate SSH key

ssh-keygen -t rsa -C "your_email@example.com"

Enter file in which to save the key (/c/Users/you/.ssh/id_rsa): [Press enter]

Enter passphrase (empty for no passphrase): [Type a passphrase]
Enter same passphrase again: [Type passphrase again]

# start the ssh-agent in the background
ssh-agent -s
# add the key
ssh-add ~/.ssh/id_rsa

Git, for example:
https://help.github.com/articles/generating-ssh-keys/
http://www.ece.uci.edu/~chou/ssh-key.html

_______________________________________

SSH on command line


Using a private key file

ssh -i [path to key file][user]@[host]

On Windows you'll have to install SSH command line tool:

http://www.windows-commandline.com/windows-command-line-ssh/

More:
 
http://www.cyberciti.biz/faq/force-ssh-client-to-use-given-private-key-identity-file/

http://support.suso.com/supki/SSH_Tutorial_for_Linux

_______________________________________

SSH to Linux Machine with Putty

Download putty from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

Verify the integrity of the file with SHA2 (preferably)

Double click on Putty.exe to run.

Enter IP address and choose SSH as shown below.

The machine you are connecting to should probably require additional configuration to ensure your session is encrypted. 

 

For example, connecting to AWS instances will require you to provide the key used when you started the instance:

AWS SSH:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstancesLinux.html

Putty uses ppk format for files.

You may need to generate a key, or convert a key using puttygen (download on putty site).

 

_______________________________________

Make a SHA1 hash of a string


echo -n some_text_here | sha1sum | awk '{print $1}'
_______________________________________

Make a SHA2 hash of a string 

echo -n some_text_here | sha256sum 
_______________________________________

Create Sha1 hash of a file

sha1sum [file] > [file].sha1
_______________________________________

Verify Sha1 hash of a file
 
sha1sum -c [file].sha1
_______________________________________

Mount CDRom

If there's a CD Rom on your machine and need to mount it:

mount cdrom 

Or 

mount /dev/cdrom

Or

mount /mnt/cdrom

_______________________________________

Mount an ISO

If you have an iso you want to mount onto a directory:

sudo mount -o loop [iso] [directory] 

_______________________________________

Eject CD Rom

eject

_______________________________________

Determine if ASCII file is Unix or Windows

Both Windows and Unix ASCII files use a carriage return at the end of a line. If you use the file command on a Unix ASCII file it will tell you the file is an ASCII file. If you use the file command on a Windows ASCII file it will tell you the file is an ASCII file with CRLS line terminators.

file [file name]
_______________________________________

dos2unix and unix2dos

The first utility transforms DOS files to unix and vice versa for the second.

dos2unix [file] 
unix2dos [file]

_______________________________________

Convert binary file to hex and other formats
Hexdump

https://www.suse.com/communities/conversations/making-sense-hexdump/

hexdump -x

and

od -x

Both produce same output of binary to hex

_______________________________________

view shell command history (.bash_history)

view recent commands

history

find specific text in history with grep

history | grep pwd

Search the .bash_history file. Get the folder where the .bash_history file is stored from the HIST environment variable. Go to that folder.

cat .bash_history | grep "some value to find"

_______________________________________

Log files on Linux

Log files are typically in this directory: /var/log

Logs are generally created by rsyslog:

/etc/rsyslog.d/
/etc/rsyslog.conf

Types of logs:

messages - application messages
system -  system messages
access.log - apache access log

Many applications specify their own application specific log locations

Browsers history shows what pages have been visited
_______________________________________

Proxy Settings on Linux

Different apps have proxies set in configuration files or different places on Linux.

User specific proxy settings for Firefox on Linux: click edit > preferences > advanced > network > settings

Package managers generally have their own specifics for setting a proxy.

Proxy server applications such as SQUID can also be set up on Linux to cache and proxy requests

Different versions of Linux set proxies in different ways. There may be a gui for this (*gasp*). Otherwise the network configuration is generally stored in a file with some variable for the proxy. [More later if time]

Command line proxy settings:
http://www.shellhacks.com/en/HowTo-Use-a-Proxy-on-the-Linux-Command-Line

_______________________________________

Find a Deleted File

lsof

_______________________________________

Find Process Using a File

lsof [Name of file]

_______________________________________

Find Script That Started a Process

ps
_______________________________________

Bring back deleted file

Get the source code for a running process running contents of a deleted file:

lsof | grep myfile

cp /proc/[pid]/fd/[file descriptor] myfile.saved

http://archive09.linux.com/feature/58142

_______________________________________

Environment Variables

set, printenv, env

get environment variables for a process

xargs --null --max-args=1 < /proc/[pid]/environ

About environment variables:

https://www.digitalocean.com/community/tutorials/how-to-read-and-set-environmental-and-shell-variables-on-a-linux-vps

Environment variables for a process:

http://serverfault.com/questions/66363/environment-variables-of-a-running-process-on-unix

_______________________________________

Show Libraries used by an application

user@home ~/ $ ldd [path to application]

http://en.wikipedia.org/wiki/Ldd_%28Unix%29

_______________________________________

TCP Wrappers

TCP Wrappers wraps network access controls around applications.

Configuration files: 

/etc/hosts.allow
/etc/hosts.deny
 
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Security_Guide/sect-Security_Guide-TCP_Wrappers_and_xinetd-TCP_Wrappers_Configuration_Files.html

Find out if a file is compiled with TCP Wrappers

http://www.cyberciti.biz/faq/tcp-wrappers-hosts-allow-deny-tutorial/
 

Essential VMWare

Set Up VMWare Player

Download VMWare player for free at VMWare.com

Get an ISO that contains an OS that can be run as a virtual Machine. If you don't know how or where to get this refer to the simple steps at the bottom of this post for a Linux ISO 

http://websitenotebook.blogspot.com/2014/08/installing-fedora-in-vmware-fusion-to.html

Run VMWare Player.

Click Player in top menu, then File, then New Virtual Machine.

Click Installer disc image file (ISO):

Click browse to select your ISO.

Click next.

Select the correct OS and version.

Name your machine whatever you want.

I chose default for the rest of the options.

Your VM is now in the list of available VMs to "play" in VMWare.

Play Your VM

Click on your VM in the list. On the right side click "Play Virtual Machine"

Stop your VM

Click on Player at the top. Then Power. Then suspend to pick up where you left off later or shut down to completely stop the VM.

Get your mouse back after clicking in VM

Ctrl-Alt

Edit VM Settings 

Get to settings for a VM that is not running: click on VM in list and then "edit virtual machine settings"

Get settings for VM that is running: Ctrl-Alt to get your mouse. Click down arrow next to Player in top menu, then Manage, then Virtual Machine settings.

You can also use the Ctrl-D shortcut to get to settings for running VM (after Ctrl-Alt if you have clicked in the VM)

Error starting VM 

PXE-E53: Check virtual machine settings. Make sure it is using the correct OS and version for your ISO and the correct ISO file is selected. Double check all settings. For example you need to choose 32 bit OS from drop down for a 32 bit ISO.

Networking Adapters 

When you install VMWare on windows you will see two new network adapters. These support the different types on network access allowed for your VM.

Network Options 

Change these in your VM settings. Go to Settings. Then click Network Adapter on Hardware tab.

Bridged VM has full network access via host machine using host's Ethernet adapter 

https://www.vmware.com/support/ws5/doc/ws_net_configurations_bridged.html

NAT uses tne network adapter named VMNet8.

https://www.vmware.com/support/ws3/doc/ws32_network21.html

Host Only No network access outside the host machine. Uses network adapter named VMNet1.

Custom Customized network setting 

VMWare Tools

Once you have your VM up and running install VMWare tools on it if you want to do things like copy from the VM terminal to the guest machine. 

Click on Player, Manage, then Install VMWare Tools

Follow the instructions. There is a link to more help.

Determine the IP address of your VM

Linux: ifconfig

Windows: ipconfig

Determining the IP of your Host

Use the same as above for the host but note the main adapter IP address.

Test connectivity

From your host, ping the VM IP and vice versa.

Ping [ip]

After pinging use arp to see the correct IP and MAC address for the VM got in your arp cache.

arp -a

Troubleshoot Bridged Mode

Go to VMNet8 adapter as described above.

Hard code an IP address.

Disable adapters you are not using if they are getting selected anywhere.

In VM network settings where you selected bridged mode click advanced and select the main network adapter for your host.

Troubleshoot Host Only

Make sure the VMNet01 adapter is set to DHCP or the VM won't be able to reach the network.

Snapshots

One cool feature in VMWare you only get if you upgrade to Workstation (on Windows) is taking snapshots. Snapshots can be taken at points in time while you are working with a VM and then you can revert to that snapshot if necessary. This is useful when you are testing malware that might
 destroy a VM, or if you are setting up a new VM or working on some project and you don't want to lose the state of your project at any point. If something goes wrong, just revert to a prior snapshot.

For example I was setting up Kali Linux and something got hung up. I had to restart the VM and lost a bunch of changes. This prompted me to get VMWare Workstation...

Checking it out...

Setting up Domain Name on AWS Route 53

If you want to host a web site on AWS, the first thing you need to do is get your domain name set up. There are basically three steps to this process:

1. Register a domain name (if you need a new one). 
2. Setup DNS records in Route 53.
3. Tell the registrar what DNS servers to use for your domain.  

DNS just tells computers on the Internet where to find your web site. If you want to know more read this: http://en.wikipedia.org/wiki/Domain_Name_System

Amazon's DNS service is called Route 53 It has 100% up time guarantee: 

http://aws.amazon.com/route53/sla/

There are two options for registering a new domain and setting up in route 53:

1. Register the domain name with Amazon

2 Register with a third party service and tell the other company to use Amazon servers for DNS.

Option 1: Register Domain Name with Amazon

1. Log into Amazon and click on Route 53

2.Click "Registered Domains" on left. Then click register domain



3. Follow the instructions to enter contact information and register the domain

4. At the end of this process the domain is under "pending registrations". It took less than 30 minutes for my domain registration to complete.

5. Once complete, the domain showed up under "Registered Domains" and  DNS servers were immediately associated with my domain.



6. Click "Manage DNS"



7. Click on the domain name you just registered in the list of  "Hosted Zones". Note that there is a comment that says this entry was created automatically by Route 53.



8. Follow the instructions below to set up DNS entries to point your domain name to your web server. See: Set up DNS records on Route 53

Option 2: Register your domain name with a third party

1. Register your domain with a third party registrar such as https://www.networksolutions.com/ and https://www.godaddy.com/

2. During or after registration, or for an existing domain name, you will need to tell the registrar which DNS servers to use for your domain (what was done automatically when the domain name was registered with Amazon in step 5 above).

3. To obtain AWS DNS server information - Click on Hosted Zones on Left and Create Hosted Zone:



3. On the right side of the screen enter the domain name and click "Create".




4. After you click "create" (or if you click the name of your hosted zone in the list of hosted zones you created) you'll get a screen which gives you four DNS server names. These are the server names you'll need to enter at the third party registrar to associate your domain name with AWS Route 53 DNS servers.



5. Follow the instructions in the next section to associate your domain name with your web server.

Set up DNS records on Route 53

At this point you have a domain name that belongs to you. You either registered at Amazon and the domain was automatically associated with DNS servers, or you associated your domain with the AWS Route 53 DNS servers by entering them at your third party registrar.

Now you need to edit your DNS records to tell the world what IP address to go to in order to see your web site.

1. Within Route 53 in the AWS console click on "Hosted Zones".

2. Click on your domain name and then "Go To Record Sets".



3. For a new domain you will see the associated name servers (NS record) and an SOA record:



4. Add an "A" record and specify which IP address someone on the Internet should go to in order to see your web site. Click "Create Record Set". Leave Type = A (default), enter an IP address (e.g. an elastic IP pointed to something hosted at Amazon or the IP address of a server not hosted by another company) and click "create".




5. In the example above I associated the domain nebulous.ventures with an IP address. I probably also want people to get to my web site if they type in www.nebulous.ventures so I will create an A record for that the same way except that I specify "www" in front of the domain name.



6. EMAIL: If you want to have email addresses associated with this domain you'll need to set up "MX" records. For example if you are using gmail you would get the MX records from them when you set up your service with Google and plug them in here to tell the world to send email from this domain name to the gmail mail servers.

7. SPAM: If you set up email you will want to set up an "SPF" record to tell the world which IP addresses are allowed to send email for this domain. This tells people who receive email from your domain if it is valid or not. If you do not have SPF records set up or they are incorrect your mail may go to spam folders. SPF records are beyond the scope of this blog post - your email provider should tell you how to set these up.

8. CNAME: For some Google services they ask you to enter a CNAME record to prove you own the domain you are trying to use with their services. This is where you would enter that CNAME record.
 

FYIs:

It will take some time for your web site to be visible at your domain, because the changes you put into Route 53 have to be propagated to all the DNS servers around the world.

You can transfer a domain you already registered to Amazon Route 53 but this is not required to use the service.

The instructions above also work with domains you already have registered.

For some registrars, when you change DNS records, they take your site offline for a period of time until the DNS entries update to the new DNS servers, so you may want to create the AWS records first, and get the DNS servers to enter while you are registering the domain.

I found some types of domains to be cheaper at my existing registrar, some cheaper on Amazon.

If you host your domain at Amazon and give someone access to Route 53, they could have the ability to transfer your domain away from you. Make sure you set up permissions appropriately in the console. Registering at a third party and not giving the people who manage things in the AWS console also prevents this.

Multi-factor authentication for your AWS account is, as always, recommended to protect assets accessible in the AWS console.

A domain name registered with Amazon initially has the status "client transfer prohibited". This will hopefully go away in 60 days.