Archive
Automate Remote SSH Control of Computers with Expect Scripts
How to Combine Expect & Bash Scripts
Full Tutorial: https://nulb.app/x6vub
Subscribe to Null Byte: https://goo.gl/J6wEnH
Kody’s Twitter: https://twitter.com/KodyKinzieCyber Weapons Lab, Episode 210
Bash scripts are the normal way to get into automation. However, they have their limitations. In this episode of Cyber Weapons Lab, we’ll look at those limitations and learn about an alternative called expect scripts. Which, can be useful when we need to respond to variables, such as when you log in via SSH.
To learn more, check out the article on Null Byte’s site: https://nulb.app/x6vub
Automate tasks with Bash scripts: https://youtu.be/PPQ8m8xQAs8
Automate recon with Bash scripts: https://youtu.be/keK99avGLvQFollow Null Byte on:
Twitter: https://twitter.com/nullbyte
Flipboard: https://flip.it/3.Gf_0
Website: https://null-byte.com
Vimeo: https://vimeo.com/channels/nullbyte
How to automate shell scripts with expect command
25 Useful Basic Commands of APT-GET and APT-CACHE for Package Management
This article explains how quickly you can learn to install, remove, update and search software packages using apt-get and apt-cache commands from the command line. This article provides some useful commands that will help you to handle package management in Debian/Ubuntu based systems.
https://www.tecmint.com/useful-basic-commands-of-apt-get-and-apt-cache-for-package-management/
Change the last line in a file
Problem:
I needed to remove a comma from the last line of a file.
Solution:
Learnt that I can specify the last line with sed.
# File with commas $ less minerals.csv "copper", "bronze", "gold", "platinum",
$ sed '$ s/,//' minerals.csv $ less minerals.csv "copper", "bronze", "gold", "platinum"
Addresses
Sed commands can be given with no addresses, in which case the command will be executed for all input lines; with one address, in which case the command will only be executed for input lines which match that address; or with two addresses, in which case the command will be executed for all input lines which match the inclusive range of lines starting from the first address and continuing to the second address. Three things to note about address ranges: the syntax is addr1,addr2 (i.e., the addresses are separated by a comma); the line which addr1 matched will always be accepted, even if addr2 selects an earlier line; and if addr2 is a regexp, it will not be tested against the line that addr1 matched.
After the address (or address-range), and before the command, a ! may be inserted, which specifies that the command shall only be executed if the address (or address-range) does not match.
The following address types are supported:
number
Match only the specified line number.first~step
Match every step’th line starting with line first. For example, ”sed -n 1~2p” will print all the odd-numbered lines in the input stream, and the address 2~5 will match every fifth line, starting with the second. first can be zero; in this case, sed operates as if it were equal to step. (This is an extension.)
$
Match the last line./regexp/
Match lines matching the regular expression regexp.
\cregexpc
Match lines matching the regular expression regexp. The c may be any character.
GNU sed also supports some special 2-address forms:
0,addr2
Start out in “matched first address” state, until addr2 is found. This is similar to 1,addr2, except that if addr2 matches the very first line of input the 0,addr2 form will be at the end of its range, whereas the 1,addr2 form will still be at the beginning of its range. This works only when addr2 is a regular expression.
addr1,+N
Will match addr1 and the N lines following addr1.
addr1,~N
Will match addr1 and the lines following addr1 until the next line whose input line number is a multiple of N.
Source:
https://stackoverflow.com/questions/3576139/sed-remove-string-only-in-the-last-line-of-the-file
https://linux.die.net/man/1/sed
Reading environment variables from a file into a Makefile
Problem:
You have a bunch of environment variables that are explicitly set at the beginning of a Makefile. These variables are then used in various recipes in the Makefile. The issue was that when a developer runs the Makefile, these environment variables overwrite their local settings.
Solution:
One option is to move the environment variables out of the Makefile and use include to add them back. So the developer can use whatever values they have in their current setup as long as the names are the same.
# Makefile
foo=FOO1
bar=BAR1
baz=BAZ1
export foo
export bar
export baz
.PHONY: target1 target2 target3
target1:
@echo $(foo)
target2:
@echo $(bar)
target3:
@echo $(baz)
After splitting it up.
# .env file foo=FOO1 bar=BAR1 baz=BAZ1 export foo export bar export baz
# Makefile
include .env
.PHONY: target1 target2 target3
target1:
@echo $(foo)
target2:
@echo $(bar)
target3:
@echo $(baz)
The problem with this approach is that the .env file needs to exist everywhere it will run. If you have some CI/CD setup that may not be the case. Instead have the Makefile as below and have the ENV vars set up the usual way you would in your CI/CD setup. On the local machine one can simply run $ source .env before running $ make to get things to work.
# Makefile
.PHONY: target1 target2 target3
target1:
@echo $(foo)
target2:
@echo $(bar)
target3:
@echo $(baz)
Source:
https://stackoverflow.com/questions/39005789/reading-makefile-variables-from-bash-script
GNU Makefile setting env variables
Problem:
Exporting env variables in Makefiles. The issue was that the environment variables were not visible across different targets. I needed the same env vars to be seen in both target1 and target2 below.
Solution:
Export the variables outside the target, eg at the top of the file.
VALUE1="myvalue1"
VALUE2="myvalue2"
export VALUE1
export VALUE2
target1:
echo $VALUE1
target2:
echo $VALUE2
Source:
https://stackoverflow.com/questions/2826029/passing-additional-variables-from-command-line-to-make
https://www.linuxquestions.org/questions/programming-9/exporting-makefile-variables-to-%24-shell-environment-807422/
Useful Linux Command Line Bash Shortcuts You Should Know
In this article, we will share a number of Bash command-line shortcuts useful for any Linux user. These shortcuts allow you to easily and in a fast manner, perform certain activities such as accessing and running previously executed commands, opening an editor, editing/deleting/changing text on the command line, moving the cursor, controlling processes etc. on the command line.
Although this article will mostly benefit Linux beginners getting their way around with command line basics, those with intermediate skills and advanced users might also find it practically helpful. We will group the bash keyboard shortcuts according to categories as follows.
https://www.tecmint.com/linux-command-line-bash-shortcut-keys/
docker: Error response from daemon…
Problem:
I was getting this error.
docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused “exec: \”telnet\”: executable file not found in $PATH”: unknown.
$ docker run --name greeter -d --expose 5000 alpine /bin/sh -c "echo Welcome stranger | nc -lp 5000" 51b6a0b73dc1cbc9b39757dff20ada1f5638c1ec781625d9e2fc6c9cced01639
$ docker exec greeter ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:AC:11:00:02
inet addr:172.17.0.2 Bcast:172.17.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:9 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:758 (758.0 B) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
$ docker run alpine telnet 172.17.0.2 5000 docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"telnet\": executable file not found in $PATH": unknown. ERRO[0000] error waiting for container: context canceled
$ docker run -it alpine /bin/sh / # telnet 172.17.0.2 5000 /bin/sh: telnet: not found
Solution:
This error means that whatever I am trying to run does not exist. So my options are to install telnet in the container or use a different command like `nc`.
port forwarding
Problem:
You want to access a site with a particular IP. Eg google maps only allows a set of IPs to access. So running code from a vagrant box then accessing from the host is not so straight forward.
Solution:
Went with using ssh
$ ssh -nNT -L 3000:localhost:3000 vagrant@127.0.0.1 -p 2222
So on the host I can access a service running on a vagrant box (192.168.33.10:3000) via localhost:3000
find command examples
Problem:
I needed to clean up after running tests
Solution:
Using find
$ tree
.
├── bin
│  └── hello
├── hello
│  ├── hello.py
│  └── __init__.py
├── requirements.txt
├── setup.py
└── tests
├── __init__.py
└── test_hello.py
3 directories, 7 files
$ python3 setup.py test running test running egg_info creating hello.egg-info writing dependency_links to hello.egg-info/dependency_links.txt writing top-level names to hello.egg-info/top_level.txt writing hello.egg-info/PKG-INFO writing manifest file 'hello.egg-info/SOURCES.txt' reading manifest file 'hello.egg-info/SOURCES.txt' writing manifest file 'hello.egg-info/SOURCES.txt' running build_ext test_print_hello (tests.test_hello.TestHello) ... ok ---------------------------------------------------------------------- Ran 1 test in 0.014s OK
Now more files are created.
$ tree
.
├── bin
│  └── hello
├── hello
│  ├── hello.py
│  ├── __init__.py
│  └── __pycache__
│  ├── hello.cpython-35.pyc
│  └── __init__.cpython-35.pyc
├── hello.egg-info
│  ├── dependency_links.txt
│  ├── PKG-INFO
│  ├── SOURCES.txt
│  └── top_level.txt
├── requirements.txt
├── setup.py
└── tests
├── __init__.py
├── __pycache__
│  ├── __init__.cpython-35.pyc
│  └── test_hello.cpython-35.pyc
└── test_hello.py
6 directories, 15 files
To delete the *.pyc files and the __pycache__ and hello.egg-info folders, run the following commands.
$ find . -type d -name "hello.egg-info" | xargs rm -rf $ find . -type d -name "__pycache__" | xargs rm -rf $ find . -name "*.pyc" | xargs rm -f
Note:
$ find --version find (GNU findutils) 4.7.0-git Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Written by Eric B. Decker, James Youngman, and Kevin Dalley. Features enabled: D_TYPE O_NOFOLLOW(enabled) LEAF_OPTIMISATION FTS(FTS_CWDFD) CBO(level=2)
pseudo-scripter 