Back to home

INF226 notes - Software Security

March 02, 2019

Mathias Bøe

Learning goals

  • Be able to explain the most common weaknesses in software.
  • Can use tools to discover security problems in software.
  • Be able to asses the security of given source code or application.
  • Know the best practises to defend software systems.
  • Understand security issues related to system development.
  • Know different techniques to avoid security risks/problems.

Learning materials

  • Books:
    • Secure and resilient software development, Merkow
    • Security for software engineers, Helfrich


30% of the grade is based on the mandatory assignment.

  • Assignment 1: Buffer overflow and SQL injection, 15th of September
  • Assignment 2: Security analysis of software, 13th of October
  • Assignment 3: Writing security critical code. 10th of November

You are expected to do these during the group sessions.

Two approaches to software security

What are the holes in the program?

  • Promotes adhoc solutions, might be hard to find all the security holes. It’s hard to find all the holes if you don’t know where all the holes are.

How to design the program securely?

  • Buildt with security in mind all the way, this way you wont get “as many” holes to patch. This is more of a preventive approach.

Terms and definitions

  • Software security: Is the ability of software to function according to intentions in an adverserial environment.


Prove that every triangle is a polygon

A logical argument has three parts:

  • Conclutions
  • Assumptions
  • Deduction

Assumptions Security mechanisms Security requirements

  • Identify security requirements: which capture the intentions for the software.
  • Make explicit the assumptions: about the environment the software will run.
  • Design mechanisms: which satisfy the requrements given the assumptions.

Exercise - Instant messenger, requirements

  • End to end encryption
    • Assumption: Secure key exchange.
    • Keyed hashfunction (signature)
  • Only sender and recipient get access to the message
    • Authentication of users, password, hashed and salted.
  • Authentication of message integrity, make sure the message is unaltered on the way to the user.
  • Authentication of recipient, you should not be able to alter the recipient after the message is sent.
  • Sidechannel attacks might be a problem (attacker takes control of the application itself and not the transferred data between sender and recipient).

Examples of mechanisms

  • Choice of programming language
  • Rate limiting
  • Sanity checks on user inputs

Examples of assumptions

  • User input cannot be trusted to have property X.
  • IP adresses can be spoofed.
  • Computer resources are finite.
  • Developers write bugs.


Website should have a high uptime- High performance language (Assumption: Able to handle the time cost)Server has limited capacity to precess requests. An attacker could send a lot of requests.
 - IP based rate limiting 
 - User prioritizing 
 - Captchas 
 - Require proof of work, for each request. 

How to prevent legit users experiencing downtime because of an attacker?

Vulnerabilities and exploits

  • The most serious vulnerabilities lead to the attacker being able to run any code on the victim machine.

The Morris worm

In 1998, a worm exploiting a buffer overflow in fingerd spread across the internet.

  • A bug in the code made the virus’s host grind to a halt.
  • Internet was partitioned for several days during the cleanup.

Even today, twenty years later, buffer-overflow exploits remain some of the more common vulnerabilities. The attacker is now a professor at MIT.

Buffer overflow

#include <studio.h>

int main(int argc, char* argv) {
  char buffer[8];
  int a = 3;
  fgets(buffer, 256, stdin);
  printf("You entered: %s \n", buffer);
  printf("and a = %i \n", a);

Buffer overread

The simplest mistake one can make in an unsafe language is reading outside the bounds of a buffer (array).

  • Example: The “Heartbleed” bug in libssl was caused by not bounds checking the TLS heart-beat signal before responding.

TODO The call stack

TODO Shell-code

The easiest exploit a buffer overflow bug:

  • Fill the buffer with attack code.


Return Oriented Programming

Return Oriented Programming (ROP) is an exploit technique using preexisting code in the program or libraries instead of uploaded shell code.


How to prevent catastrophic failure?

  • Write better C code.
  • Static analysis.
  • Stack canaries.
  • W^X
  • Adress space layout randomization.
  1. Stack Canaries

    A stack canary is a random integer value written after the function return pointer on the stack. When the function returns the integer value is checked to detect if it (and thus the return pointer) has been overwritten since function call initiated.

  2. TODO Rearranging the stack

    To prevent important values

  3. TODO W^X

    Memory allocations can give the allocated memory different properties:

    • Writable
    • Executable


  4. Prevention

    Best practice to avoid buffer overflows:

    • Use memory safe languages.
    • Use memory-safe abstractions in unsafe languages (say vectors or smart pointers in C++)
    • Use the compiler’s abilities.
    • Run static analysers to identify potential bugs.
  5. Example

    The code in afunction could access:

    • Arguments from the caller: f(x,y,z)
    • Local variables.
    • Global variables.
  6. Breaking memory safety

    • Pointer arithmetic
    • Unconstrained casting
    • No bounds-check on array access
    • Unsafe de-allocation

TODO Languages

We can distinguish between memory safe languages and languages with direct access to pointer arithmetic. Memory safe:

  • Java/C# (Bounds check on arrays, runs on virtual machine).
  • Most scripting languages (Python, JS)
  • more

Achieving memory safety

Automated memory management:

  • Garbage collection (LISP, Java, Haskell, Go, )
  • Resource allocation is initialisation (RAII) and borrows checker (Rust).

Undefined behaviour

TODO Examples of undefined behaviour

  • Undefined behaviour: is code which