seminar/paper.txt

Overview of IoT Fuzzing Techniques
Tuan-Dat Tran
University of Duisburg-Essen
tuan-dat.tran@stud.uni-due.de

Abstract—Due to the rising popularity of IoT
devices and embedded systems and their usage in
not only in the business sector but also at home,
the focus has been shifting on the security of those
devices. To address this issue, there have been
many approaches in detecting, analyzing and mitigating security flaws in IoT devices like static[8]
and dynamic analysis[9]. Another approach to vulnerability detection is fuzzing.
Fuzzing is a technique originally used for automated black box testing software and became a
highly researched topic[6][37][35][36], expanding its
usage from black box testing to white and gray box
testing. Fuzzers generate test cases to test software
for vulnerabilities. The generation of those test
cases are done in many different ways.
IoT fuzzers focus on fuzzing IoT devices. Although there are similarities to regular fuzzing,
fuzzing IoT devices comes with its own constraints
and techniques.
In this paper we are comparing techniques used
by IoT fuzzers to circumvent the challenges presented by IoT devices and the constraints of the
solutions proposed by the IoT fuzzers.

I. Introduction
Internet of Things (IoT) devices and embedded
systems are becoming more and more prevalent,
and with billions of devices being connected to
the internet, they are an integral part of everyday
life[16]. Despite IoT devices being so widespread
they are riddled with security vulnerabilities,
which makes them an easy target for attackers,
since many of those vulnerabilities are considered
“low-hanging fruits”[6]. One example of such a
vulnerability in IoT devices is the 2016 Mirai
botnet which consisted of an average of 200.000 to
300.000 IoT devices[3] while it was suspected that
over half a million are vulnerable to the security
vulnerabilities the Mirai Botnet utilized[19].

While information leakage[38] and insecure login credentials[3] are just some of the many security problems an IoT device can have, detection
and mitigation of these security flaws has proven
itself to be challenging. One approach to discover
those flaws is called fuzz-testing, or fuzzing.
Fuzzing is a method to test software for flaws
by automatically generating and sending large
amount of malformed data to the software. This
is done while the fuzzer monitors the softwares
reaction to this data for malfunction like crashes
or other unexpected behavior. The goal of fuzzers
is to detect vulnerabilities in the software in an
automated manner. Despite the simplistic approach to vulnerability detection it has proven
itself to be effective[21]. The simplistic approach
enabled researchers to extend the capabilities
of fuzzing from creating test cases consisting of
random data to sophisticated systems which use
a wide variety of information about the context
and source code of the software. The complexity
of fuzzing tools goes so far that the techniques
used by fuzzers diverge to suche an extent that
they can be divided into different classes, each
with their own strength and weaknesses[22].
A huge advantage of fuzzing compared to other
vulnerability detection approaches is the automation of the analysis, which makes fuzzing a highly
scalable method to find vulnerabilities[23]. Scaling up the fuzzing process enables a higher rate
of generation test cases and therefore increases
the coverage on the target program. On the other
hand, insufficiently “smart” test generation methods can lead the fuzzer to run for a long time
without finding any new code paths or otherwise
advancing the fuzzing process[28].
IoT fuzzing applies the methodologies of
fuzzing to IoT devices. Just like different kind of

fuzzers, IoT fuzzing has its own advantages and
disadvantages. IoT devices offer a large surface
area regarding communication e.g. network protocols, their companion app or their web interface[6][5][35]. For this reason fuzzers which were
not originally designed to fuzz IoT devices can
still be utilized for IoT fuzzing, like in the case
of boofuzz, which was developed with the intent
to fuzz network protocols[5]. IoT Fuzzing also
opens the door for new techniques, unique to
IoT devices, by fuzzing the companion app of the
device[6].
In this paper we present an overview of different
fuzzing tools and techniques for IoT devices. We
focus on advantages and disadvantages of those
techniques in context of their use-case to help
developers and researchers find the right tool for
their job and weigh in the positive and negative
aspects of existing approaches. The IoT fuzzing
tools chosen for the overview, were chosen to
cover as many different recently developed fuzzing
techniques to the best of our abilities.
The paper is structured as follows. First we
introduce IoT devices, firmware and general
fuzzing. In Section III we lead into IoT fuzzing
and its challenges to create a knowledge basis to
introduce IoT fuzzing techniques in Section IV,
the main section. Section V contains related work,
that is closly tied to iot fuzzing. And finishing up
with the conclusion in section VI.

Home network
User

PC

Router

Smartphone

Printer

Smartwatch

Zigbee/Z-Wave

IoT Hub
Smart Lightbulb

Internet

Figure 1. Example of IoT home network (inspred by Wang et
al.[35]).

The wide applicability of IoT devices in the
context of business, manufacturing and home-use
increases the surface area for vulnerabilities to
be found. IoT devices, being so diverse regarding their functionalities and ways to offer their
services, further increases the possible ways to
accumulate vulnerabilities[38].
IoT devices, due to being built for specific purposes, do not need as much processing power as a
general computer does. This leads to them having
a hardware platform specifically tailored to their
use case. And due to the heterogenic nature of
IoT devices in terms of e.g. operating systems,
instruction sets or memory layouts, analysis of
the firmware proves difficult[8]. Reasons for this
are the different requirements a manufacturer has
for the device like the energy efficiency, real-time
processing or memory footprint[14].
IoT devices, and especially home-based ones,
use multiple ways to connect to the internet. IoT
devices connect to the internet either directly
through WiFi or via an intermediary device like a
smartphone and connecting to it with Bluetooth.
Another way is having an IoT hub which acts
as proxy between other IoT devices and either
another intermediary via Bluetooth or directly
WiFi[35]. This leads to many ways an IoT network can be structured depending on the kind
and number of IoT devices (Figure 1).

II. Background
A. IoT devices and embedded systems
The terms IoT devices and embedded systems
describe a large amount of devices. Embedded
systems are devices which interact with their surroundings via sensors and regulators and are built
to serve a specific purpose[23]. IoT devices on
the other hand are broadly described as devices
which extend regular devices with an internet
connection to enable them to communicate over
the internet[27]. The term embedded devices can
describe many devices such as cameras or industrial control systems (ICS), which makes it
hard to generalize embedded devices. This also
applies to IoT devices, since the extension of
an embedded system by an internet connection,
makes it an IoT device.

2

The works of Hahm et al.[14] propose a classification into low-end and high-end IoT devices
and dividing those two classifications into three
subcategories for low-end devices. Those classes
represent the complexity and computing capability of those devices with “Class 0” having
the least resources and “Class 2” devices having
the most resources. Multi-purpose systems (i.e.
smartphones and computers) deploy many mechanisms to detect faults like segmentation faults
and report them through core dumps. IoT devices may not have such functionalities. The more
minimalistic design of IoT fuzzers causes them to
only perform the specific tasks they were built
for. Therefore functionalities like heap hardening
may not be present due to the IoT devices limited
computing power and constrained costs[23].

“blob firmware”[30], consisting of the application
and system code compiled together.
Like all software, firmware is susceptible bugs
and misconfigurations, which can lead to vulnerabilities[8]. For this reason, analysis tools are
needed to find such vulnerabilities. There are
several methods to analyze firmware for bugs, but
they all have to face the challenge of working
around the heterogenity of firmware[8].
To analyze firmware, firmware first has to be
acquired. This can be done by downloading it
from the vendors website. An alternative method
of acquiring firmware is extracing it from the
physical device. This is done by either some kind
of debugging port or by reading the flash memory
directy. Extracting firmware manually poses a
challenge in itself, since debugging ports are not
always available on the end product[6][31].
B. Firmware
Additionally firmware is often packed or even
IoT firmware on IoT devices is the intermediate encrypted, which poses as an obstacle for
layer between high higher level software and the firmware analysis. In some cases proprietary comhardware of the device. This is done by providing pression algorithms or encryption without knowla simplified interface, that can be used by such edge of the secret key makes firmware analysis
higher level software[13]. Since firmare commu- infeasible or even impossible.
nicates with many parts of the IoT device, it
C. Fuzzing
contains a lot of information about it.
There are several types of firmware based on
Fuzzing describes the process of testing a softthe type of device they are used in. In the works of ware for faulty or unexpected behavior by sending
Muench et al.[23] devices are classified in “Type- it malformed messages as input[12]. Fuzzing is
0” to “Type-III” systems:
widly used, as
T0 (Type-0) systems represent multi-purpose
There are multiple types of fuzzing techniques
systems, which don’t fall under the classification based on the amount of known information about
of embedded systems or IoT devices.
the software: Whitebox, blackbox and greybox
T1 (Type-1) devices are devices, which use a fuzzing. Whitebox fuzzing has complete informageneral purpose operating system like Linux. The tion about the software’s source code. Blackbox
operating system on T1 devices is often modified fuzzing on the other hand has no such information
to be more minimalistic and offer a lightweight while greybox fuzzing lies in between regarding
user environment like busybox.
the available information. Blackbox fuzzing relies
T2 (Type-2) devices run on completly cus- purely on the binary of a program or the program
tomized operating systems which are tailored to in its already executed state[20]. This leads to
the devices use case. In order to save space and blackbox fuzzers generally creating many unneccomputational power, typical operating system essary test cases due to the lack of knowledge
functions like a Memory Management Unit may about the internals of the target[13]. Another
be omitted.
problem with blackbox fuzzers is therefore also
T3 (Type-3) devices run on a single control the detection of errors. Internal system errors,
loop. On these devices the firmware and the which may lead to misbehavior at a later time,
software, which runs the devices functionalities, can not be easily detected by black box fuzzers
are a single instance. This leads to a so-called as they occur. Blackbox fuzzers therefore often

3

rely on externally visible exceptions. Advantages
of black box fuzzing are the narrow and quick
tests due to the limited surface area to target,
focusing only on the aspects of the software
the user interacts with[18]. Additionally blackbox fuzzing may be the only way of fuzzing a
target when there is no source code available.
Whitebox fuzzers on the other hand have access
to the source code of the fuzzing target. Test
cases generated by whitebox fuzzers are based
on the analysis of the given source code. Techniques like symbolic execution or dynamic taint
analysis are utilized to increase the efficiency of
the fuzzer. In comparison to blackbox fuzzing,
whitebox fuzzing usually has a higher overhead
since the additional analysis is performed on
the targets source code[21]. Greybox fuzzers take
the middle ground between white and black box
fuzzers and only use some information about the
internals of the target software to improve the
fuzzing process. This may be done by injecting
instrumentation to the binary at compile time[7]
or by performing lightweight static analysis on
the source code of the software[21]. The usage
of limited knowledge enables greybox fuzzers to
have higher throughput than white box fuzzers,
while being more accurate than black box fuzzers.
Comparing blackbox fuzzers with greybox fuzzers
or even whitebox fuzzers is therefore not feasable,
due to the different starting conditions and use
cases[11].
The basic fuzzing process can be divided into
three steps: (1) input generation and sending
that input to the software, (2) monitoring the
softwares behavior in reaction to the given input
and (3) adjusting the input according to the
softwares behavior (Figure 2).
During the input generation step the fuzzer
generates and prepares messages according to its
generation strategy. Choosing which generation
strategy is used depends on the given information
or constraints of the system that is being fuzzed.
The given information about the fuzzing target
differentiates fuzzers into the categories black-,
white- and greybox fuzzers.
Monitoring the softwares behavior upon receiving a malformed message as input is another
step of a typical fuzzing loop. The monitored

behavior depends on the earlier steps, but after
every loop the original programs state should be
restored to have an equal ground for all test
cases. When the fuzzer looks for XSS bugs or
SQL injections, the program will not crash, when
such a bug is triggered. This has to be taken into
consideration while monitoring the software and
therefore other methods of detecting those bugs
will have to be used than methods, which are
used to detect crashes due to memory errors like
buffer overflows. Fuzzers who do not try to trigger
crashes usually use the applications answer to the
input message to determine whether the test case
triggered the event, which was tested for[5]. To
monitor a softwares crash the fuzzer can provide
instrumentation, with which the tested software
is compiled[1], if greybox fuzzing is used. A blackbox approach could be monitoring for specific
output of the software after a malformed input
has been sent or monitoring the status of the
network connection for networking capable software. A fuzzers goal is to cover as many parts of
the software as possible. Evaluating the coverage
of the software is only possible for greybox or
whitebox fuzzers, since they can instrument the
code. This metric can be used to guide the fuzzing
process of generating input like in the state-of-art
mutational fuzzer AFL[1] and its fork afl++[2].
Another property of fuzzers is their adjustments to the input after a fuzzing loop is done.
They are categorized into smart and “dumb”
fuzzers. Dumb fuzzers are not aware of the input
structure and therefore only try random input,
substitutions based on heuristics, delete parts of
the input or add parts to the input. This can lead
to a lot of test cases, which do not lead anywhere.
Another disadvantage is that input generated
by a dumb fuzzer may easily be dismissed if a
specific input structure is expected. Looking at
smart fuzzers, which try to generate valid input
based on the softwares protocol[5], grammar[15]
or model[26]. To perform smart fuzzing the input
model must be provided to the fuzzer, which may
not be as easily accessible on proprietary devices,
although there are ways to derive an input model
from a large sample of valid and invalid input.
Advantages of fuzzing are the automation and
scalability of the process. This enables fuzzing

4

Start

Input

own multitude of challenges. The third challange
is instrumentation, used in non-blackbox fuzzing
approaches, to collect code coverage information
and detect subtle memory corruptions. There are
multiple approaches to add instrumentation to a
program or its environment for regular fuzzing.
A challenge of adding instrumentation to IoT
devices is the need to often use static and dynamic
analysis to imitate the functionalities of instrumentation, since they often can not be directly
applied to the IoT device. The reasons for this
will be explained later.
(1) The challenge of fault detection on IoT devices means, that memory corruptions in the device caused by IoT fuzzers can often go unnoticed
since they do not necessarily lead to crashes. Protection measurements on multi-purpose systems
detect memory corruptions on the system caused
by fuzzing and cause a crash, making memory corruptions therefore visible to regular fuzzers. Such
measurements are rarely implemented on IoT
devices due to limited computing resources[23].
A liveness check, also called probing, can be
performed to check the status of the device while
fuzzing it. Probing can either be active or passive.
During active probing the fuzzer sends regular
known to be valid messages to the target system
and evaluates the response. The messages sent by
the fuzzer may cause a state change in the tested
device, which has to be accounted for. Passive
probing uses the devices responses to the test
message to determine liveness or observes visible
crashes.
Muench et al.[23] expands on this by classifying system crashes by their observability. An
observable crash is therefore the most visible
and managable kind of crash. During observable
crashes the device stops running and provides
some form of error message or draws attention to
the faulty behavior in another way. It is added
that this also includes crashes, which do not
provide additional information about the crash,
such as error messages. Observable crashes are the
optimal case among system crashes, since they are
visible from the outside and enable the fuzzer or
tester to react to the crash without delay.
Reboots are another kind of crash. A crash
inducing error of a software on an IoT device

Instrumentation
A̶͛ ̡Ā̴̠
Input queue

Mutated file

File
AA

Mutation

AB

Tested
Software

Mutated file
Crash
Try next file

A7
Mutated file

Add to queue
NO

YES

New code path?

Figure 2. Fuzzing with AFL[1].

to run many test cases in a short amount of
time which makes throughput of the fuzzer an
important metric in evaluating fuzzers[7]. This is
achieved by easily being able to run software concurrently on multiple processors. An alternative
way is running the software in a virtual environment[23] and executing the virtual environments
concurrently.
III. Challenges of IoT Fuzzing
IoT fuzzing is the applicatoin of fuzzing techniques on IoT devices. This approach poses
new challenges, since fuzzing hardware and its
firmware and fuzzing software operate on different
domains, which have each their own challenges.
Muench et al.[23] describe the main challanges:
The first challange, fault detection, is about the
complexity of observing crashes on IoT devices
during the fuzzing process. Fuzzing regular software may already yield unobservable unexpected
behavior. Working with IoT devices adds another
layer to this problem, since IoT devices do not
have the same I/O capabilities and memory protection measurements as a multi-purpose system
does. The second challenge in IoT fuzzing is
the performance and scalability of the fuzzing
process. Running a regular fuzzer concurrently
on multiple processes rarely poses a challenge.
When fuzzing IoT devices, either multiple copies
of the same device have to be bought to create
a comparable scenario, which is often infeasable
or emulation has to be utilized, which poses its

5

usually does not lead to the crash of the whole
system, since they work indipendantly from each
other. In T3 devices, where the software and
firmware are one and the same, a crash of an
outward facing service leads to a crash of the
whole system.
In reaction to malformed input a device may
hang. That means that it halts execution and
does not react to any more input. This may be
due to being stuck in an infinite loop. This leads
to a slowdown in throughput of a fuzzer and the
device needs to be restarted if such behavior is
found.
Late Crashes pose a challenge for testing the
device. This behavior is described as the device
crashing after a non-negligible amount of time
after the real cause of the crash, like a malformed
message, is sent. This makes correlation between
the cause and the crash challenging.
At last there are cases where neither the device
nor the software crash while still being in an
unexpected state. This can lead to wrong data
and incorrect output. This kind of malfunctioning
of the device is hard to detect, since the fuzzer
needs information about the expected response
to determine whether its an output caused by a
malfunction or not. This is further complicated
due to the diverse message formats used in IoT
devices[11].
There are also cases of malformed input not
causing any visible effects, even when errors occurred. These errors may cause crashes or malfunctioning at a later time, which makes detecting
them during fuzzing almost impossible without
instrumentation[11].
(2) The second challenge is performance and
scalability. While regular fuzzers can execute
and test software concurrently to increase the
throughput and therefore find more possible
faults in the software over time. Fuzz testing
on an IoT device is not possible in the same
manner, since a physical device is being fuzzed.
Even though multiple copies of the same device
could be purchased, to scale up the test cases, it
would become infeasible due to financial limitations and infrastructure requirements like power
and space. Emulation can help with the problem
of scalability by emulating the test device, but

this approach faces the challenge of IoT devices
being dependent on the hardware components of
the device[39].
After a fuzzing loop, the original state of the
tested device has to be established to start every
fuzzing attempt under the same conditions. This
is not a challenge with regular software, since
the softwares original state is reestablished after
rerunning it. Changes on the file system, that
were caused by the tested software can can be
easily reverted with e.g. a snapshot of the virtual
machine running the test. To establish a testing
condition on IoT devices, without the knowledge
of it’s internals, the easiest method is restarting
an IoT device. This step can take up to a minute,
which negativly affects the throughput of IoT
fuzzers.
(3) The third challenge Muench et al.[23]
mentions is instrumentation. Instrumentation on
desktop systems is used to obtain coverage information about the software that is being fuzzed
and detect memory corruptions by adding them
during compile or run time. Instrumentation being added during compile time therefore requires
the firmware beforehand. This is already an issue
on IoT devices, since acquiring the firmware is
not always possible. Additionally the variety of
operating systems and processor architectures,
makes instrumentation on IoT devices a challenging task. Additionally, obtaining the manufacturers tool chain to re-compile the firmware with
instrumentation is rarely possible. A workaround
to this approach could be the use of binary
dynamic instrumentation frameworks like valgrind[34] or using QEMUs instrumentation[33],
but these methods heavily depend on the OS and
CPU architecture.
Furthermore, IoT fuzzing suffers from the similar or the same problems as regular fuzzing based
on the fuzzing approach. Therefore an IoT fuzzer
which utilizes network protocol fuzzing will face
the same challenges as the used network protocol
fuzzer, like generating valid input[29], on top of
the aforementioned challenges of fuzzing an IoT
device.

6

Additionally, since this is a blackbox appraoch
to input generation, coverage is another issue.
4) Snippet-based mutation: Snipped-based mutation is a novel approach to input generation of
Snipuzz[11]. Snippet-based mutation is the application of the mutation-based fuzzing approach on
snippets. Snippets are parts of messages, determined by a heuristic algorithm and hierarchical
clustering. Those snippets are categorized by the
response they trigger from the IoT device. Snippets are then used to build new messages to trigger new program states. This method of mutation
and message generation creates messages, which
more likely follow message or protocol constraints
of IoT devices, which leads to more effective
fuzzing. Since this mutation method is guided
by the response of the tested device, detailed
responses are required to accurately categorize
snippets[11].
5) Message Structure Tree: Message Structure
Tree is a mutational fuzzing technique where the
valid input is analyzed to create a tree structure
based on heuristics to mutate single fields of the
input[20]. This way the grammar of the protocol
can be derived without explicitly providing the
input format.
6) Stateful Message Generation: This technique was introduced by SIoTFuzzer[37] which
fuzzes web interfaces of IoT devices. Stateful
Message Generation (SMG) is divided into three
parts: front-end analysis, state analysis and seed
generation. SMG considers that communication
depends on certain states and therefore groups
together messages as a test case to fuzz the target
system. So far SMG is only used to fuzz web
interfaces in SIoTFuzzer[37].

IV. Overview of IoT Tools and
Techniques
We give an overview of different IoT fuzzers,
their techniques and list their advantages and
disadvantages (Table I).
A. Input Generation
1) Mutational fuzzing: Mutation based fuzzing
is a method of input generation[25]. Mutational
fuzzing requires a set of predefined messages to
start the mutation on. These mutation can include e.g. bit flipping, checking for out of bound
bugs, sending empty data or subsubstituting
parts of the message with random data[11] to
explore new program states or trigger unexpected
behavior. This way the fuzzing process can get
started easily with only a couple of, so called, seed
messages. A disadvantage of mutational fuzzing
is the limited coverage. A mutational fuzzer can
rarely generate input, that deals with a targets
complex sanity checks, since mutational input
generation does not take the input format into
account[25].
2) Generational fuzzer: Generation based
fuzzers create messages from scratch while being
provided with the format specifications for
the input. Creating such a format specification
requires manual effort and may even be infeasible,
especially if the format is not available[25]. In the
work of Srivastava et al.[32] they attribute the
performance of FirmFuzz to their generational
approach of input generation, due to resulting
the constrained state space, that leads to a
decreased overhead.
3) Under-constrained Input Generation:
Under-constrained Input Generation is a
technique utilized by the fuzzer DIANE. Here
a combination of static and synamic analysis is
used on the companion app to find functions,
that produce “valid yet under-constrained”
inputs for the IoT device. The companion apps
own functions are then used to generate input
for the IoT device, that is not constrained by
the app and structurally correct enough to not
be discarded by the IoT device. The limiations
of this approach lie in the implementation of
the app analysis to find the desired functions.

B. Instrumentation
1) Binary Rewriting/Instrumentation: Binary
rewriting can be used to add instrumentation to
firmware[23]. Instrumentation can be used to e.g.
add hooks to specific functions. This is interesting for fuzzing once instrumentation is added
to internal exceptions to check for crashes or
otherwise unexpected behavior[13]. To perform
binary rewriting disassembly of the firmware is
necessary, which requires partial decompilation.
An additional challenge is the missing room for

7

Tool

Technique

Target

Fuzzing Techniques

Crash detection

SIoTFuzzer[37]

Blackbox

Web Interface

Stateful Message Generation*

Network Monitor

IoTFuzzer[6]

Blackbox

Companion App*

Generation&Mutation
Taint analysis

Passive probing

Firm-AFL[39]

Greybox

Firmware

Mutation
Augmented Process Emulation*

Emulation

Snipuzz[11]

Blackbox

API

Snippet-based mutation*

Network Monitor

Firmcorn[13]

Greybox

Firmware

Optimal virtual execution*
Vulnerability-oriented fuzzing*

Instrumentation

Diane[29]

Blackbox

Companion App

Under-constraint Input Generation*

Passive probing
Active Probing

HFuzz[20]

Greybox

Network protocol

Message Structure Tree*

Instrumentation

WMIFuzzer[35]

Blackbox

Web Interface

Mutation

Network Monitor

* = Novel technique in fuzzer
Table I
An overview of different IoT fuzzing tools.

of fuzzers[11]. Full emulation of the firmware with
the help of heuristics mitigates the lack of fault
detection and increases accuracy of found vulnerabilities to a level of desktop system application fuzzers. Additionally emulation based fuzzing
provides the possibility to repeat test cases and
their executions to further analyze specific test
runs[24]. Often third party developers lack details
of the device to implement good emulator. This
makes building emulator requiring huge amounts
of manual effort[23]. This is due to IoT devices
being heavily dependent on their hardware[39].

additional instrumentation due to embedded devices being optimized for their memory usage[23].
2) Symbolic Execution: A technique used to
increase code coverage by using symbols as input
and tracking manipulations and comparisons of
them during runtime[36]. The usage of the input
is then backtracked to solve the constraints of
specific code branches if a desired state is reached.
Symbolic execution has the problem of path explosions and constraint solving, which poses as an
obstacle to scalability[7]. Path explosions is the
exponential increase of code branches the larger
the program is. A part of this problem are possible
infinite loops. Constraint solving can pose another
challenge, since depending on “how deep” the
programs tracking goes, the calculation of the
constraints of a specific branch can be complex.
3) Taint analysis: Taint analysis is used to
track data of interest during execution. The data
that is being tracked is called taint source. IoTFuzzer[6] uses taint analysis to track e.g. user
input to find out which input influences network
messages sent to the analyzed IoT device.

2) Partial Emulation: Partial emulation can
lead to accurate vulnerability detection with decreased performance in comparison to full emulation, but possibly better performance than
fuzzing the physical device, since it makes the
fuzzing process more scalable[23]. Partial emulation is done by only emulating parts of the
firmware or its peripheral devices.
3) Augmented Process Emulation:
This
method of emulation is proposed and used by
Firm-AFL[39]. Augmented process emulation
utilizes both system-level emulation and usermode emulation to increase execution speed of
the tested firmware or software. Here systemlevel emulation is only used when necessary,
due to its low speed, while user-mode emulation

C. Emulation
1) Full Emulation: Emulation tackles the
problems of throughput and scalability in IoT
fuzzing. This is done by improving the performance, success rate and hardware-indipendance

8

is used the rest of the time. This improves
the overall throughput of fuzzers utilizing
Augmented Process Emulation compared to
fuzzers using emulators, that only make use of
system emulation. Currently Augmented Process
Emulation is limited to firmware that can be
emulated in a system emulator and runs a
POSIX-compatible operating system.
4) Optimized Virtual Execution: This technique used by Firmcorn[13], where the firmware
instructions are executed in a lightweight CPU
emulator. This approach circumvents the overhead generated by full-system emulation. The
execution is further optimized by using heuristic
algorithms like omitting unnecessary functions to
optimize the execution process.

E. Crash Detection

1) Active Probing: Active probing is used to
determine the state of the target by regularily
sending messages to the target. The response of
the target to such a message is known. Should
the response deviate from the expected message
or should the device not respond at all, it can be
assumed that there is an error.
While this probing method can detect errors
that do not lead to crashes, the probing messages
could lead to unexpected states of the target
themselves. Sending additional messages to probe
for the liveness of the target, also slows down
the overall fuzzing process, since such probing
messages do not contribute to the coverage of the
target.
2) Passive Probing: During passive probing
D. Code Coverage
the messages, that are sent for fuzz testing are
used to determine the state of the target. While
1) Vulnerability-oriented
fuzzing:
Vulnerability-oriented fuzzing is used in the target device responds in a acceptable time
Firmcorn[13]. For this method, static analysis window, it it assumed, that no crash has occured.
is used to find vulnerable code. Vulnerable
V. Related Work
code is determined by multiple factors like,
complexity, number of memory operations and A. Static Analysis
call to sensitive functions. Those attributes are
Alternatively to fuzzing there are other ways
calculated based on information about the the to test software for vulnerabilities such as static
targets control flow, like the number of edges firmware analysis. Static firmware analysis is the
of a function or the cyclomatic complexity of a analysis of firmware without executing it by using
function.
tools like binwalk[4] to unpack the firmware and
2) Coverage-oriented
fuzzing:
Coverage- reverse engineering it with a reverse engineering
oriented fuzzing generates input with the tool like IDA[17][10]. The advantage of static
traversion of different execution paths in mind. analysis is the possibility to automate and scale
This is done to maximize code coverage to reach the processes of analyzing the firmware[8], since
paths which may be vulnerable by taking the the testing does not depend on a physical device.
ability of an input to trigger new paths into On the other hand, static analysis also yields a
account[28]. While coverage guided fuzzing tries high amount of false positives and may not find
to maximize code coverage, usually most of a completely new vulnerabilities with the usage of
softwares code is not vulnerable, therefore a lot its heuristics[39]. Another challenge during static
of resources are spent on exploring invulnerable analysis is the handling of packed or obfuscated
code paths.
code, since it first has to be unpacked or deobfus3) Directed fuzzing: Direct fuzzing is the pro- cated to perform meaningful analysis on it[8].
cess of generating input with the goal of travers1) Dynamic Analysis: Dynamic Firmware
ing specific execution paths[28]. Since only a analysis is another alternative to fuzzing. For
fraction of firmware code has vulnerabilities the dynamic analysis the firmware is executed to be
graybox approach to fuzzing by increasing code investigated. This can be done in a multitude of
coverage leads to test cases, which end up not ways. For example by running the firmware on the
finding vulnerabilities[13].
original device or emulating the device to have

9

the firmware run in a virtual environment. The
running firmwares behavior is then analyzed[9].
The challenge of working with packed or obfuscated code during static firmware analysis can be
overcome with dynamic analysis[36] by emulating
the physical device, which increases scalability
and eliminates the need to acquire the physical
device to test it[9].

[8]

[9]

VI. Conclusion
In this paper we created an overview of the
different IoT fuzzing techniques used by state
of the art IoT fuzzing tools and compared their
approaches in regards to input generation, crash
detection and their device scopes. The IoT fuzzers
we looked at, utilized many techniques to make
use of many attack surfaces and even used software outside the IoT devices themselves to gain
information about the device, like IoTFuzzer[6].

[10]

References
[1]
[2]
[3]

[4]
[5]
[6]

[7]

american fuzzy lob. https://github.com/google/AFL.
American Fuzzy Lop plus plus.
https://github.com/AFLplusplus/AFLplusplus.
Manos Antonakakis et al.
“Understanding the Mirai Botnet”. In: 26th USENIX
Security Symposium (USENIX Security 17).
Vancouver, BC: USENIX Association, Aug. 2017,
pp. 1093–1110. isbn: 978-1-931971-40-9. url:
https://www.usenix.org/conference/usenixsecurity17/
technical-sessions/presentation/antonakakis.
Binwalk. https://github.com/ReFirmLabs/binwalk.
boofuzz. https://github.com/jtpereyda/boofuzz.
Jiongyi Chen et al. “IoTFuzzer: Discovering Memory
Corruptions in IoT Through App-based Fuzzing”.
In: 25th Annual Network and Distributed System
Security Symposium, NDSS 2018, San Diego,
California, USA, February 18-21, 2018.
The Internet Society, 2018.
url: http://wp.internetsociety.org/ndss/wpcontent/uploads/sites/25/2018/02/ndss2018%5C_01A1%5C_Chen%5C_paper.pdf.
Peng Chen and Hao Chen.
“Angora: Efficient Fuzzing by Principled Search”.
In: 2018 IEEE Symposium on Security and Privacy, SP
2018, Proceedings, 21-23 May 2018, San Francisco,
California, USA. IEEE Computer Society, 2018,
pp. 711–725. doi: 10.1109/SP.2018.00046.
url: https://doi.org/10.1109/SP.2018.00046.

[11]

[12]

[13]

[14]

[15]

10

Andrei Costin, Jonas Zaddach, Aurélien Francillon, and
Davide Balzarotti. “A Large-Scale Analysis of the
Security of Embedded Firmwares”.
In: Proceedings of the 23rd USENIX Security
Symposium, San Diego, CA, USA, August 20-22, 2014.
Ed. by Kevin Fu and Jaeyeon Jung.
USENIX Association, 2014, pp. 95–110. url:
https://www.usenix.org/conference/usenixsecurity14/
technical-sessions/presentation/costin.
Andrei Costin, Apostolis Zarras, and
Aurélien Francillon.
“Automated Dynamic Firmware Analysis at Scale: A
Case Study on Embedded Web Interfaces”.
In: Proceedings of the 11th ACM on Asia Conference on
Computer and Communications Security, AsiaCCS
2016, Xi’an, China, May 30 - June 3, 2016. Ed. by
Xiaofeng Chen, XiaoFeng Wang, and Xinyi Huang.
ACM, 2016, pp. 437–448.
doi: 10.1145/2897845.2897900.
url: https://doi.org/10.1145/2897845.2897900.
Yaniv David, Nimrod Partush, and Eran Yahav.
“FirmUp: Precise Static Detection of Common
Vulnerabilities in Firmware”.
In: Proceedings of the Twenty-Third International
Conference on Architectural Support for Programming
Languages and Operating Systems, ASPLOS 2018,
Williamsburg, VA, USA, March 24-28, 2018.
Ed. by Xipeng Shen, James Tuck, Ricardo Bianchini,
and Vivek Sarkar. ACM, 2018, pp. 392–404.
doi: 10.1145/3173162.3177157.
url: https://doi.org/10.1145/3173162.3177157.
Xiaotao Feng et al. “Snipuzz: Black-box Fuzzing of IoT
Firmware via Message Snippet Inference”.
In: CoRR abs/2105.05445 (2021). arXiv: 2105.05445.
url: https://arxiv.org/abs/2105.05445.
The OWASP Foundation. Fuzzing | OWASP. 2021.
url:
https://web.archive.org/web/20210414111843/https:
//owasp.org/www-community/Fuzzing (visited on
04/14/2021).
Zhijie Gui, Hui Shu, Fei Kang, and Xiaobing Xiong.
“FIRMCORN: Vulnerability-Oriented Fuzzing of IoT
Firmware via Optimized Virtual Execution”.
In: IEEE Access 8 (2020), pp. 29826–29841.
doi: 10.1109/ACCESS.2020.2973043.
url: https://doi.org/10.1109/ACCESS.2020.2973043.
Oliver Hahm, Emmanuel Baccelli, Hauke Petersen, and
Nicolas Tsiftes. “Operating Systems for Low-End
Devices in the Internet of Things: A Survey”.
In: IEEE Internet Things J. 3.5 (2016), pp. 720–734.
doi: 10.1109/JIOT.2015.2505901.
url: https://doi.org/10.1109/JIOT.2015.2505901.
Renáta Hodován, Ákos Kiss, and Tibor Gyimóthy.
“Grammarinator: a grammar-based open source
fuzzer”. In: Proceedings of the 9th ACM SIGSOFT
International Workshop on Automating TEST Case
Design, Selection, and Evaluation,
A-TEST@ESEC/SIGSOFT FSE 2018, Lake Buena
Vista, FL, USA, November 05, 2018. Ed. by
Wishnu Prasetya, Tanja E. J. Vos, and Sinem Getir.
ACM, 2018, pp. 45–48. doi: 10.1145/3278186.3278193.
url: https://doi.org/10.1145/3278186.3278193.

[16]

[17]
[18]

[19]

[20]

[21]

[22]

[23]

[24]
[25]

[26]

[27]

[28]

Mark Hung. “Leading the IoT Gartner Insight on How
to Lead in a Cnnected World”.
In: Gartner Research 1 (2017), pp. 1–5.
IDA Pro. https://hex-rays.com/ida-pro/.
Mohd Ehmer Khan, Farmeena Khan, et al.
“A comparative study of white box, black box and grey
box testing techniques”.
In: Int. J. Adv. Comput. Sci. Appl 3.6 (2012).
Eduard Kovacs.
Over 500,000 IoT Devices Vulnerable to Mirai Botnet.
2016. url:
https://web.archive.org/web/20210507170030/https:
//www.securityweek.com/over-500000-iot-devicesvulnerable-mirai-botnet (visited on 05/07/2021).
Xinyao Liu, Baojiang Cui, Junsong Fu, and Jinxin Ma.
“HFuzz: Towards automatic fuzzing testing of NB-IoT
core network protocols implementations”. In: Future
Gener. Comput. Syst. 108 (2020), pp. 390–400.
doi: 10.1016/j.future.2019.12.032.
url: https://doi.org/10.1016/j.future.2019.12.032.
Valentin Jean Marie Manès et al. “The Art, Science,
and Engineering of Fuzzing: A Survey”. In: IEEE
Transactions on Software Engineering (2019), pp. 1–1.
doi: 10.1109/TSE.2019.2946563.
Richard McNally, Ken Yiu, Duncan Grove, and
Damien Gerhardy. Fuzzing: the state of the art.
Tech. rep. DEFENCE SCIENCE and TECHNOLOGY
ORGANISATION EDINBURGH (AUSTRALIA),
2012.
Marius Muench, Jan Stijohann, Frank Kargl,
Aurélien Francillon, and Davide Balzarotti.
“What You Corrupt Is Not What You Crash:
Challenges in Fuzzing Embedded Devices”.
In: 25th Annual Network and Distributed System
Security Symposium, NDSS 2018, San Diego,
California, USA, February 18-21, 2018.
The Internet Society, 2018.
url: http://wp.internetsociety.org/ndss/wpcontent/uploads/sites/25/2018/02/ndss2018%5C_01A4%5C_Muench%5C_paper.pdf.
Panda. https://github.com/panda-re/panda.
Hui Peng, Yan Shoshitaishvili, and Mathias Payer.
“T-Fuzz: Fuzzing by Program Transformation”.
In: 2018 IEEE Symposium on Security and Privacy, SP
2018, Proceedings, 21-23 May 2018, San Francisco,
California, USA. IEEE Computer Society, 2018,
pp. 697–710. doi: 10.1109/SP.2018.00056.
url: https://doi.org/10.1109/SP.2018.00056.
Van-Thuan Pham, Marcel Böhme, and
Abhik Roychoudhury. “Model-Based Whitebox Fuzzing
for Program Binaries”.
In: Proceedings of the 31st IEEE/ACM International
Conference on Automated Software Engineering.
ASE 2016. Singapore, Singapore: Association for
Computing Machinery, 2016, pp. 543–553.
isbn: 9781450338455. doi: 10.1145/2970276.2970316.
url: https://doi.org/10.1145/2970276.2970316.
Brien Posey. IoT devices. 2021. url:
https://web.archive.org/web/20210520072243/https://
internetofthingsagenda.techtarget.com/definition/IoTdevice (visited on 05/20/2021).
Sanjay Rawat, Vivek Jain, Ashish Kumar,
Lucian Cojocar, Cristiano Giuffrida, and Herbert Bos.

[29]

[30]

[31]

[32]

[33]
[34]
[35]

[36]

[37]

11

“VUzzer: Application-aware Evolutionary Fuzzing”.
In: 24th Annual Network and Distributed System
Security Symposium, NDSS 2017, San Diego,
California, USA, February 26 - March 1, 2017.
The Internet Society, 2017.
url: https://www.ndss-symposium.org/ndss2017/ndss2017-programme/vuzzer-application-awareevolutionary-fuzzing/.
Nilo Redini et al.
“DIANE: Identifying Fuzzing Triggers in Apps to
Generate Under-constrained Inputs for IoT Devices”.
In: 42nd IEEE Symposium on Security and Privacy
2021. 2021.
Nilo Redini et al. “Karonte: Detecting Insecure
Multi-binary Interactions in Embedded Firmware”.
In: 2020 IEEE Symposium on Security and Privacy, SP
2020, San Francisco, CA, USA, May 18-21, 2020.
IEEE, 2020, pp. 1544–1561.
doi: 10.1109/SP40000.2020.00036.
url: https://doi.org/10.1109/SP40000.2020.00036.
Nilo Redini et al. “Karonte: Detecting Insecure
Multi-binary Interactions in Embedded Firmware”.
In: 2020 IEEE Symposium on Security and Privacy, SP
2020, San Francisco, CA, USA, May 18-21, 2020.
IEEE, 2020, pp. 1544–1561.
doi: 10.1109/SP40000.2020.00036.
url: https://doi.org/10.1109/SP40000.2020.00036.
Prashast Srivastava, Hui Peng, Jiahao Li,
Hamed Okhravi, Howard E. Shrobe, and
Mathias Payer. “FirmFuzz: Automated IoT Firmware
Introspection and Analysis”.
In: Proceedings of the 2nd International ACM Workshop
on Security and Privacy for the Internet-of-Things, IoT
S&P@CCS 2019, London, UK, November 15, 2019.
Ed. by Peng Liu and Yuqing Zhang. ACM, 2019,
pp. 15–21. doi: 10.1145/3338507.3358616.
url: https://doi.org/10.1145/3338507.3358616.
TriforceAFL.
https://github.com/nccgroup/TriforceAFL.
Valgrind. https://www.valgrind.org/.
Dong Wang, Xiaosong Zhang, Ting Chen, and
Jingwei Li. “Discovering Vulnerabilities in COTS IoT
Devices through Blackbox Fuzzing Web Management
Interface”. In: Secur. Commun. Networks 2019 (2019),
5076324:1–5076324:19. doi: 10.1155/2019/5076324.
url: https://doi.org/10.1155/2019/5076324.
Jonas Zaddach, Luca Bruno, Aurélien Francillon, and
Davide Balzarotti.
“AVATAR: A Framework to Support Dynamic Security
Analysis of Embedded Systems’ Firmwares”.
In: 21st Annual Network and Distributed System
Security Symposium, NDSS 2014, San Diego,
California, USA, February 23-26, 2014.
The Internet Society, 2014. url:
https://www.ndss-symposium.org/ndss2014/avatarframework-support-dynamic-security-analysisembedded-systems-firmwares.
Hangwei Zhang, Kai Lu, Xu Zhou, Qidi Yin,
Pengfei Wang, and Tai Yue.
“SIoTFuzzer: Fuzzing Web Interface in IoT Firmware
via Stateful Message Generation”.
In: Applied Sciences 11.7 (2021), p. 3120.

[38]

[39]

C. SSH Brute-force

Nan Zhang et al. “Understanding IoT Security
Through the Data Crystal Ball: Where We Are Now
and Where We Are Going to Be”.
In: CoRR abs/1703.09809 (2017). arXiv: 1703.09809.
url: http://arxiv.org/abs/1703.09809.
Yaowen Zheng, Ali Davanian, Heng Yin,
Chengyu Song, Hongsong Zhu, and Limin Sun.
“FIRM-AFL: High-Throughput Greybox Fuzzing of
IoT Firmware via Augmented Process Emulation”. In:
28th USENIX Security Symposium, USENIX Security
2019, Santa Clara, CA, USA, August 14-16, 2019.
Ed. by Nadia Heninger and Patrick Traynor.
USENIX Association, 2019, pp. 1099–1114.
url: https://www.usenix.org/conference/
usenixsecurity19/presentation/zheng.

For brute forcing the SSH login for “iotgoatuser” I used hydra. An alternative to brute forcing over ssh would be getting the /etc/shadow
and /etc/passwd files and cracking the passwords
from there locally with tools like JohnTheRipper or Hashcat. This method would circumvent
defense mechanisms like fail2ban, although the
usage of such defense mechanisms is unlikly on
an IoT device.
0

hydra -l iotgoatuser -P ./data/passwords.txt ssh

VII. Appendix
://TARGET -t 4 -f
1
A. Reconnaissance
To gain information about the system we start
Figure 5. Brute-force ssh
off with nmap. The result of the port scan resulted
in 6 open ports: 22 (ssh), 53 (dns), 80 (http), 443
(https), 5515 (unknown) and 65534 (unknown).
Knowing that there was a backdoor service on D. MITM
this device it was probably on either port 5515
When visiting the webinterface of the IoT deor 65534, since those are not part of the IANA vice via Firefox, we are greeted with a warning,
well-known ports.
that the certificate is self-signed. This poses a
threat to the user, since self-signed certificates
0
nmap $TARGET -pcan not be revoced and don’t expire. If the cer1
nmap $TARGET -sV -sC -p22,53,80,443,5515, 65534
tificate was somehow leaked, the integrity of the
2
website could not be restored without replacing
Figure 3. SYN scan all ports and detailed scan over open ports. the certificate, which may not be easily done on
an IoT device sold to hundreds or thousands of
consumers. Self-signed certificates are used nonB. Getting shell and adding user
theless on IoT devices, since they are easier to
Connecting with port 5515 via netcat returned obtain and free of charge.
a root user shell. To add a user I simply edited the
To proceed with the testing I had to simply
/etc/shadow and /etc/passwd file by adding one press “Accept the Risk and Continue” in the
entry in each file. The entry for the /etc/passwd- browser.
file contained the username, uid etc. and the
Logging in on the web interface with the creother one contained the username, md5 hashed
dentials we obtained in the brute forcing step
password etc. for the /etc/shadow-file. To check
didn’t seem to have worked. I then tried bruteforcing the login form with burpsuites “Intruder”
0
echo "echo tuan:x:1001:1001::/root:/bin/ash >> /
function and a wordlist from SecLists1 , using the
etc/passwd;exit" | nc -nv $TARGET 5515
1
echo ’echo tuan:\$1\$123456\
root-user and the iotgoatuser-user, which didn’t
$qqQvjw0PqIk7otmzNsUIN0:18145:0:99999:7::: >> /
work either.
etc/shadow;exit’ | nc -nv $TARGET 5515
Using the backdoor I then changed the root
2
password to “asdfasdf”, since the already existing
Figure 4. Adding user “tuan” with the password “password”. password didn’t seem easily crackable. Logging in
with the new credentials worked.
whether the user was added correctly a I logged
1
https://github.com/danielmiessler/SecLists
in via SSH with the new user.

12

Looking at the luci-directory in /usr/lib/lua/luci/
we
found
/usr/lib/lua/luci/controller/iotgoat/iotgoat.lua, which lists the secret
developer page under https://$TARGET/cgibin/luci/admin/iotgoat/cmdinject.
E. Static analysis
To start the static analysis we first extract and
unpack the filesystem by finding the filesystem in
the firmware with binwalk, extracting it with dd
and unpacking it with unsquashfs. This gives us
access to the whole file system of the IoT device
we are analyzing.
0
1
2

binwalk ./data/Syssec\ IoT\ Device.bin
dd if=data/Syssec\ IoT\ Device.bin of=data/0
x1F5A50 bs=1 skip=2054736 count=2813038
unsquashfs data/0x1F5A50

3

Figure 6. Extracting and unpacking filesystem

To find the shadow and passwd file, we can
simply run a find command to look for them or
by simply knowing, that they are usually in the
/etc/ directory.
The same can be done for the certificate to find
a certificate in /etc/ssl/certs/ca-certificates.crt.
F. Write up
The full write up can be found
here:
https://git.uni-due.de/sktatran/
syssec-embedded-security-writeup/-/blob/
main/writeup.org

13